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CHAPTER I , • 
... . . . 

PR0BIM STUDIED ■ ■ 

. Summary 

Currenl educational literatxire is filled ^vith statements that are: 
(1) exposing existing problems that are now facing education-, (2) recog- 
nizing the importance of iiproyed changes in the field of education, (3) 
si5}porting career education as a possible solutipri, and (4) promising an^ 
increase of loca J, ^tate, and f ederal^ enphasis for the approach. Consid- . 
ering that traditibnal methods of evaluation for iiino^/ative educational 
programs- such as career education have in the past failed in their attempt 
to truly assess program effectiveness (Cuba, 1973 • pp. 1 and 2), increased 
efforts need to be directed toivard the development of arf^b^luation system 
to. measure the effectiveness of school- based career ediicatioit programs. 

The major purpose of- this study is to develop an evaluation design 
to measure the effectiveness of school.- based career educ^tidn programs. 
Objectives of the study include the identificatipn of tasks and consid- 
erations involved in developing and adniinistering the evaluation design, >■ 
e.g., identification of variables, methods and instruments for collecting 
information, sources of metliods to analyze and treat data, and criteria 

for ^assessing th^ effectiveness of the evaluation design. - . 

■\ t ' ' ' - I 

The initial activitief undertaken in tTie study consisted of an exten- 
sive review of available literature pertaining to career education and 
evaluation. The procedures utilized in developing the evaluation design 
and in, identifying the tasks and considerations involved in administering 
it consisted primarily of the processes of modification, integration, and 
synthesis of models and methodologies that were currently being used by 
evaluation specialists dn tiie field of educatixjn. These integrated models 



and mejthQdologies were composited with special consideration given to. 
their adaptability to the four stages of program improvement f br^ feareer 
.education » that .we/e identified by. the researcher, * M advisory^ committee 
was established early in the study and was, utilized .throughout its 
develoiJment . ^' / . . - ' ' 

The final results of this study consist. of: (1) an , explicit defini- 
tion of evaluation, (2) an integrated eyaluati&n design correlating four 

• — "*«» * * 

stages of , evaluation with the four stages of program improvement identi- 

fied for career education, (3) suggested variables of , information which 
may be needed for the four stages of eyaluition, (4) suggested research • 
designs, types of measm;^ment devices and criterion measures, (5) suggested 
data collection methods andTt^hniques , (6) suggested commercial and non- 
commercial instruments for collecting information, (73 suggested sources ^ 
of methods which may be useful" in analyzing 'and treating information, 
(8) ^suggested procedures and formats which may be used to functionally 
feedback the resists of evaluation, and (9) suggested criteria for assess- 
ing the adequacy of the evaluation design. 

The evaluation system is intended for the use of assisting ].ocal 
agencies in evaluating the effectiveness of their school -based career 

education programs. . , \ 

> ' « , . 

X- 

" • . statement of the Problem \ , 

The central problems of this study are to: (1) develop an evalua- 
tion system to measure the effectiveness of school -based career education 
programs and (2) report the evaluation system in a functional form that 
will be useful in assisting local agencies in evaluating the effectiveness 
of their school -based career education programs. 
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Objectives for the Study ^* ^ - 

The following* objectives have been identified as the research goals 
for the study: . ' ' ^ 

1. Identify an^ define t;he nature of evaluation. . 

2. Design an evaluation system for school -based career education 
progiiams , * ^ ^ - 

3. Identify tasks and considerations involved' in developing 'and 
administering the evaluation design. 

3.1 Identify and develop vatiables which may serve as supq[€f&- 
tipns of informations needed in evaluating career education 

^ programs* 

3.2 Identify methods and instrum.ents which may be useful in 
collecting data related to the variables of information 
needed. - ^ - 

3.3 Identify sources and methods which may ser/e as references 
and suggestions to analyze and treat data. 

' 3.4, Identify procedures ^d foiinats which may serve as sug- 
gestions to feedback the resiilts of evaluation^ in a \ 
functional fom. 

3/5 Identify criteria for assessing the adequacy" of the eval- 
uation system. / 

^4. -toply the evaluation systan to a selected career education pro- 
gr^ and report the results. 

This report deals with the second major goal of a research project 
foimerly proposed for the Center for Vocational, Technical and Adult Edu- 
Cation at the University of Wisconsin-Stout by Dr. Orville Nelsoi^, Co- 
\ Director. The two major problems of concern to Dr. Nelson ^s original 
I proposal were: (1) to determine the status of, career education in Wis- 
consin schools aaid (2) to develop an evaluation system £or career educa- 
tion programs. The -first major goal has already been resear^ched and^ 

reported by Dr. Mehar Arora, research dire.ctor, in a separate issue titXed 

% j 
"An Assessment of the Status of Career Education in Wisconsin," dated, 

' ' / 

September 30, 1973. ^ , ' \ I 

^ 1-3 ' 
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Rationale for the Stu<fy 

As will be emphasized in greater detail throughout this report,^ 
evaluatiOT must be cx)nsidered as a discipline \Aose structure and content 
.is highly dependent upon the specific n:;^ire of the program \Aiich it is 
to assess. For the present time, however, it should suffice to say that 
the focus of the evaluation design to be presented in this report is 
especially geared toward assessing K-12 school -based career educatj.on 
programs. ' - . , 

Before^ one can fully recognize the need to develop effective evalu- 
ation for school -based career education programs, the following major 
factors should first be acknowledged: 

1 . There exists nation-wide dissatisfaction with present traditional 
fbims of educational systems. 

2. If is feasible to believe that much of that dissatisfaction can 
be attributed to a lack of educational emphasis in providing 
course relevancy in terms of self-discoveiy and career orient- 
ation, planning, and developnent. ... 

3. The nature of the career education approach is to provide course 
relevancy in terms of self -discovery and career orientation, 
planning, and development. 

4. There already exists nation-wide commitment and support for 
career education as a potential solution to present educational 
ills. There also is a promising increase of local, stat^ and 
federal emphasis for the approach. 

i 

c 5. 'Traditional methods of evaluation for innovative programs such 
^ career education have in the past failed in their attenpts 
to'^trlily assess program effectiveness. 

6» Career education is an innovative approach which has yet been 
left unproven. . - < 

7. In order to help determine the real utility and promise that 
' . career education prdgrams may have to offer, incTeased effort 
nfeeds to be directed toward the development of a more effective 
evaluation design. 
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The "seven major factors listed above provide the basic logic and 
foundation upon which the ronaining portion of the rationale will be 
presented. For a moi^ detailed outline of the rationale, refer to the 
one given in the table of contents. 

Dissatisfaction i%ith Present Traditional Forms of Educational Systems 

"Education has lorg been given credit for much of the greatness of 
our nation, and is now increasingly under attack for existing ills," 
CKorizek, 1971, p. 21). A review of literature reveals that a great 
deal of dissatisfaction exists on the part of ^ the American society to- 
ward the inabilities of present educational systems in meeting the grovr- 
ing needs of large numbers of American youth. Such dissatisfaction seems 
quite peculiar for a "nation whose economy has produced wealth beyond the 
imagination of any people in history" (Hoyt et al, 1972, pp. 17 and 18)/ 
Educational and social ills have been so frequently remarked upon by 
authorities that 'it "is no (longer a) secret that growing numbers have 
become disenchanted with today's schools" (Meyer, 1972, a, p.^1). 

''Students, parents, teachers- business and civic groups are call- 
ing for new ways of applying resources to educational needs . . . 
in altering "obsolete roles ♦ . . within school systems in direc- 
tions that will facilitate more creative and effective resolution . 
of educational problems" (Holstein, 1971, p. 5). 



Existing Problems Nov/ Facing Education! Current educational literature 

/ 

is filled with statements exposing *the existing problems that are now 
'facing traditional forms of education. In typical schools across the 
nation, too many students are "tuning out" because they're being "tum,ed 
off" by educational curriculums that are "dull and irrelevant" (Meyer, 
1972 a, p. 31; Herr, 1972, p. 63; Holstein^ 1971, p. 6;- Richardson, 1972^ 



( 

'1* 



(p..l), -i.e,^". . . too many while still trying to play the game feel lost 
an^lack identity" G^leyer, 1972 a, p. 31), For the student, there seems 
to exist little or no apparent relationship bet\s-een what they are asked 
■ . to learri in*^the sch?^;^ and its application to the real wrld that exists 

.outside the walls of the classroom CHolstein, 1971, p. 6; Tyler, 1970, 
Sp- 794-795; Bottoms, et al, 1972, p. 3; Keller, 1972," pp. 2 and 3). 
The following comnents by Bottoms (no date listed, p. 4) express this 
quite well: 

"For large numbers of American youth, the public school system 
represents a maze of meaningless activity leading nowhere. They 
fail' to see the relationship bet\v'een their current school exper- 
' iences and som.e identifiable next step. - MVhat has algebra to do^ . 
withm.e?' they ask, 'WTiy should I tr>' to rmoiibef the chaef bat- - 
ties of the Revolutionary War?' ■ Even the high school science 
laboratory appears to be a place for -following directions o± tne 
laboratory m^ual to see if they can obtain results reported in 
the textbook" (Tyler, 1970, pp. 794-79S). 

The following words -by Marland (1972 c, p'.- 5) seem^ appropriate at this ' 

■ point: 'T/e fail our' young people when we do not at least alert them to 

the potentialities for the training they have acquired in the academy." 

Magnitude and Rahdfications of Educational Problems . Findings that il- 
lustrate the magnitude and ramifidations of the problems just mentioned 
can also be readily documented. For example, the U. S. Office of Educa- 
tion published a study by Grant (19.65, back cover) which indicates that , 
out of every ten pupils in the fifth grade^in 1957-58, 9.4 entered the 
ninth grade, in 1961-62; 8U1 entered the eleventh grade in 1963-64; 7_.l 
grated from hi^h school in 1965; 3.8 were anticipated to enroll in 
. college in the'fall of 1965; 1.9 would have probably earned baccalaureate, 
degrees in 1969. According to these' statistics, about 30 percent of all 
youth in the U^. fail to leave high school with a diploma. Consequently, 
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they too often enter what Eaddy (1571, p. 3) called "the category of tlw 
educationally disadvantaged/' The statistics for Grant *s study included 
students with average and above average intelligence ♦ Therefore, one 
might suspect that the failure to educate 30 percent of all high school 
students in the U. "is not due priinarily to the inadequacies of the 
students, but to the inappropriateness of the program to supply them jwith 
- the kind of learning required" (Tyler, 1970, pp. 794 and 795) • 

The speech entitled "Education and Business- -A Necessary Merger" 

/ ' 

given by Sidney P.torland (1971 b, pp. 20 and 21), S. Commissioner 

/ 

of Education, is another source illustrating the magnitude of th? dilemma 

^ • / 

' education is now facing* According to Marland, education represents out 



Nation* s largest expenditure, costing $85 billion a year. It pvfen exceeds 




:vi'ould seem appropriate for taxpayers to be able to assume that educatioij 

. ' / 

is providing almost all young people vdth career skills that would at 

iSast enable them to be economically independent when the/ 'decide to leave 



school; ' ' ; 

According to Marland, however, the : • 7 

"statistics do not support this assumption. Of 3.?' million youaig 
people leaving fonnal edixation in 1970-71, nearly/2.5 million 
lacked skills adequate to enter th^ labor force at/ a level com- 
mensurate with their academic and intellectual precise. Ibny'^ 
left with no marketable skill v/hatsoever^'* / 

Out of these 2.5 million students, 850,000 ^roppq^d out of elementary 
or secondary school, 750,000 graduated from the general high school cur- 
riculum, and 850,000 entered college but dropped out without a degree or 
completion of an organized career program. In considering the costs in- 
volved in getting these three groups of students as far up the educational 



ladder as they got before leaving school, Marland estimated a combined 
outlay of almost $28 billion which, .in essence, represents approximately 



one-third of the entire funds expended oi\ education during the 1970-71 
school year. ' Thanks, to Marland, the nature of ^ucation/s dilemma is now 
clearer: "IVe spend billions to prepare 2,5 million young people for poten- 

w t 

tial disejjchantment, aimlessness and failure, year after year, after yearl" 

'^Jn light of an appropriation such as this, i^t is no^ surprising to find ^ 

I 

throughout the nation, widespread reluctance on the part of taxpayers to 
approve educational budgets (Korizek, 1971, p. 21), . ' ' * 

Even more distressing, however, are the non-monetary losses--the ^ 
uncalculated wastes of. personal an^ social investments incurred through 
the insufficient utilization and development of human resources. The loss 
of such intangibles as self- identity, self-estean, prestige, and confidence 
and the feelings df alienation and non-fulfillment that are acconpanied 
with truancy and misunderstanding are all irretrievable forfeitures that 
cannot be estimated in terms of money alone. The. dismal consequences of 
educational by-products such a/)these are, of dourse, the potential pred- 
ecessors of such social ills as unqmployment, crime, and drug addiction 
which, in turn, create the need for remedial, correctional, and welfare 
programs Vihich, in essence, represent an extra bprden upon our resources. 
In other words, the' shortcomings of ediacation have created a cycle of a 
degenerative nature which, in effect, is wasting human potential kt the 
added expense of the taxpayer. *The costs, the blighted lives, the dis- 
content, the violence and the threat of revolution, are needless^ Schools 
can prepare young people to realize their potential'* (National Advisory 
Council oulfocational Education, 1969). 
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What does the future promise i£ present trends continue? Uhlesr. 
iiprovement occurs in our educational system now, the disenchantment aiid 
waste that now exists will only perpetuate itself* In 1975, it is expected 
that the unskilled will account for less than 5 percent of the total labor ' 
force; ^however^ the Bureau of Labor Statistics anticipates ''that we will 
still have more than 3.5 millibn young people with no sala^)le skills try- 
ing to scpaeeze , themselves into this sad 5 percent category"^ (Marland, 1972 
.a^^p. 4*5).; In addition, the National Center for Education (Keller, 1972, 
pp. 4"^ and 5) predicts that '*by 1980, approximately 15 million *more people, 
most of whom will be young, idll want to enter the labor force, at a time 
when 100 million Americans wil] be seeking work." For these persons. * < 
"there will be literally no room at the bottom. This tragic situation 
clealrly indicates thatWefica's educational efforts are failing, or at 
least they are not attuned to the realities of our times" (>!arland, 1972 
a, p. 45)e , . ' 

' . • . ( . ' 

Possible Causes for Existing Ills , ^ 

N 

Societal BnpTiasis on College Preparation . The heart of the problem under 
lying education's failing efforts can be found in the "false societal 
attitude that worships a college^ degree as the best and surest route to . 
occupational success". (Hoyt, et al, 1972, p. 29).' Traditionally, schoo] 
systems, espetially the secondary schools of the nation, have placed pri- 
mary anphas is on college preparation (Drier, 1972 a, p. 5; Korizek;^ 1971 ^ 
p. 21; Jenkins, 1971, p.' 8; Keller, 1972, pp. 1 and 2). As Hoyt (et al, 
1972, p. 29) and Keller (1972, pp. 1-3) reveal, this traditional emphasis 
is founde'd on the principle that the more education a person has, the 
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better o££ ho is. The only problem with education's application o£ this 
principle, according to Hoyt and Keller, is that some o£ its basic limita 
tions and assumptions have been dang<^ously misconstrued. Some o£ the 
distinctions that education has £ailed to take into full consideration 
are: * 

1. 'Hie optimum educational preparation required for successful 
job performance differs from one occupation to another, i^e., 

' ^ot all occupations require a minimum of a college educ^-ion^ 
for a maximum development of successful job performance skills, 

2. The optimM\educational preparation for, successful job per- 
formance differs from one person to another within an occupa- 
tion depending upon the particular person's predisposition 
to success for his chosen occupation. 

3. The ampunt of formal educatioijial preparation required for 
successful* job entry is not in itself valid criteria for 
judging the worth of an occupation such criteria is more 
properly established idthin th^ontexts of both an indivi- 
dual and societal frame of reference, i.e,, the value of an 
occupation is mbre appropriately ascertained by its contj^i^u- 
tion to both the indivdual worker and the society that is to 
be served. 



4. Differing degrees of expertise and job satisfaction, ranging 
from optimum job competency development to development of 
minimum job entry skills, can be attained in any' of the pro- 
fessions and should be fiilly acknowledged, regardless of 
whether it required twelve years of fopial education as a 
prerequisite to develop or t\v*enty year's. 

5. Whatever occupation is suitably selected to keep a person's 
.vocational needs satisfied, these same vocational needs will 

still be satisfied, regardless of how much formal education 
Was required to prepare Kijn for successful performance within 
that particular occupation. / 

6. Preference of how much educational preparation one chooses 
depends largely upon the individual's particular perception 
of work and self-concept which, in turn,, has ^a direct bear- 
ing on the unique vocational needs and aspirations which differ 
widely from person to person. 
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Many persons, for reasons o£ their 'own, may be willing 
aind completely satisfied to discontinue their foniial 
educational preparation before reachingt optimum job 
competency development*, e.g., the; point on the contin- 
uum, (note Figure I) where only miriimum'-job eiitry skills 
are. develQped; v/hile still others may seek more f6rmal 
education within the same opdipation. 

the ultimate degree of job competency development also 
depends a great deal upon ^he restriction popularly re- 
ferred to as reaching one^s .own "level of incompetence" 
which is governed by the "Peter Principle" (Peter, 1969). 



Units of Formal 
Educatipnal Pre- 
patation in Years 



Continuum Depicting a Lifetime 
Figure I 
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Elementary 



If 



Jr. Sr. 
High High 



Post-Secondary 



'Optimuia Job 
Cotapecency 
Devc3opiaent: 



7y Not all 'jobs are easily marketed --no matter how much fopial 
educational preparation a person has, his chances of finding 
satisfactory employment within his chosen field are slim if 
there nd longer ^exists any marketable demand for his occupa- 
tional services. \ ^ ^ 

The failure to see clearly and account for these seven distinctions 
points to the roots of the problems that are now facing education. The 
inappropriate application ot the principle regarding , education's relation - 
ship to work preparation has resulted in a false societal attitude that 
has over-emphasized a college education and. neglected the virtue of its 
many other alternatives. Such over-emphasis has been carried out to 
"the extent that those yojuths who do not pursue it (a college education) 
are often considered 'second class"' (Hoyt, et al, 1972, p. 30). Keller' 
(1972, p. 3) aisq agrees with this i4ea as indicated by the following 



ERIC 



i-ii 
21 



f 

comments: 'These attitudes tend to* stigmatize both individuals and pro- 
gjams that have goals related to the #W0rl^' of "work, with the exception 
of those occupations classified as the professions.? In addition to 
thiSi' -the • " 

• • • 

"inordinate empliksis pn college preparatory programs, when com- ' 
bined vdth parental obsession to see that their children receive 
a college education, has created a substantial imbalance ii\ the 
. ' ' 'labor force df ou;r'N^tion" (Jenkijis, 1971, p» 8). / ' * ' * 

NIarland (1972 a, p.' 45) states that by the time students receive 

their baccalauteate- degree, 

■ - • ' f . 

*'it may well be a sui^plu^ item. We'^have an ekcess of such de- 
grees now in 'the aerospace industry and d?! certain parts of the 
teaching profession, and the National Planning Association pre- 
dicts an ^ eventual excess of bachelor's degrees in every field ^ 
except the health professions. The Department of Labor indicates 
that in the near future, .80 percent of ail jobs will be within 
the range of the high school^ diploma." 

Yfct as Bottoms (et al, 1972, p. 3) observed, "at the secondary level, 
over 80 percent of students are enrolled in either the college prfepara- 
tory or general 'education curriculum designed to ready tHem for college 
attendance." The implications for^our present forms of educational sys- 
terns are clear: *l^en over 80 percent of the students are getting ready 

to do what oyer 80 percent will not,/^in fact, actually do, something is 

»* 

surely wrong with American education" (Bottoms, et al, 1972, p. 3),. 

Tlxpre §e€Siis to be no doubt in Hoyt's (et ,al, 1972, p. 30) mind where much 

of the responsibility rests, for the societal attitude which over-emphasizes 

college preparation: " * ' ' 

'The Americaii system of formal education has, in its basic struc- 
ture, philpsophy, and methodology, been a major contributor to 
this societal attitude and must accept a major share of the blame 
for the unfavorable personal and social impacts. If American edu- 
^ cation can ^jd will changb. It can become a major contributor 
toward the goal cf eradicating this attitude in our society." • 

' ', ^ " . / 1-12 
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''As Hoyt (et al, 197'2, p; 30) points; out, changes such as these, that 
.are directed to;^^rd educational systems, are not intended to lower the 
values of a college education for those persons who have selected a col- 
lege program as a pertinent gpal in their life; as Jenkins (1971, p* 8) , 
asserts, "th^e will always be societaYdeirands ior college graduates." 
For such persons- as these, hoxvqver, it is necessary that they Ibok upon 
their college education as contributing to some relevant and personal 
goal. In addition, it Js not the intention of such charges t9 ini^ly 
that the only objective of fonnaa education i^ job preparation alone. 
Rather, it '"is a plea for acceptance of a greater variety in life styles 
and in types of career preparation." It also represents the belief that / 
formal education would have more meaning if it were not completely con- 
sidered an end in itself, i.e. /something that, for some Vague reason, 
has to be done and finished as soon as possible. ^ ' , 

Education's Emphasis: Preparation for Living vs. Preparation for ?^lalcing 
a Living . According to Hoyt (et al, 1972, p. 30) and Bottoma- (et al, 
1972, pp. 2 and 3), the two basic purposes of education have historically 
and philosophically been: (1) preparing people fcpr living and (2) pre- 
paring people for making a. living. They also assert that education has 
been primarily eiip,hasizing preparation for living rather than preparation ^ 
for niaking a living. This en?)hasis could veiy possibly be the inherent ^ 
consequence of education's response to the societal attitude earlier men- 
tioned (Keller,* 1972, pp. 1-3). As. Hoyt (1972, p.^35) states, 'The .schools 
have done quite well what weSxave tpld them' to do. .," it's just that/ 
"we have not insisted they do a number of things which are important to 
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society." In the process o£ placing major emphasis on preparation for 
living, education can, ia effect, become an end in it self • Hoyt and, 

•-Bottoms both have stated similarly that "in practice, education's prime 

** 

purpose has seemed to be education,"' as though it were the sole purpose 
of the typical classroom teacher to prepare his students for ascending 
to the next grade level in order that even bigger doses of education . 
could be admiiiistered, Hoyt appropriately referred to this emphasis in 
American education a^ "school for schooling Is sajce." .Goldhammer and 
Taylor, (1972, p. 21) expressed it tiiis way: "The 'schools are kteeped ^ 
in the academic traditions which emphasize scholarship, not effective- 
ness, in perfoiming one's routes." According to their interpretation, 
traditional curriculum, "relates tq the structure of knowledge, not to 
theslife needs of students*" 

Deficiencies in Career Planning and Development , Due to technological 
^progress and its accompanying specialization, there. have evolved more 
edi^ational and occupational opportunities than have ever existed before • 
Consequently, there now exists "unprecedented pressures, • , for indivi- ' 
duals to^efine vJ^they are and what, goals they. value" (Drier, 1972 a, 
p* 3), Hoyt (1972, p* 182) seelns to agree with this, as t}ie following 
comments clearly indicate: 'The needs of this society, both youth's and 
adylt's, in these times for individualized and continuing assistance in 
career preparation and development are great and are becoming greater," 
In addition to the understanding thai students need to define themselves 
and their jgoals in terms pf available career opportunities, studies show, 
that there exists a very high level of student interest in the areas of 



1-14 



choice-making aiid career development from the very early agfes throughout: 
the life-process of maturity (Sloccum and Bowles, 1967; Campbell, 1968) • 

Despite this need for student^ care.er developnent activities/ and a 
high level of student interest in these areas, there exists evidence 
within these same studies that^ also indicates that students are not getting 
the educational assistance they need in order* to develop the career, plan* 
ning skills and th^ personal and social clarifications that are essential 
^ for career deyelopmentT Similar studies by Shill (1968 a, 1968 b, 1968 c) 
and Eoykin (1968) , hot only support this id^ but further suggest that 
students get more career information outside the school setting than from 
within* In addition, McMinri (1971, p* 11) asserts that, "Traditional currie- 
, ulums have not assisted individuals to perceive work as having personal ^ 
relevance, as being critical to one^s deteimining^ his own life style, or 
as^bping a means that contributes to self -fulfillment." Consequently,, 
many youDh are not only isolating themselves from any meaningful partic- 
ipation in the academics of school because of boredom (a's earlier depicted 
in the section of the rationale titled 'TExisting Problems Now Facing Edu- 

♦ 

cation") , but they are also finding themselves unable to take advantage 
of the vast amount of infomatibn about caree;- opportunities that are now 
available (Drier, 1972 a, p. 3; McMiriny 1971, p. 12; Holstein, 197i; p. 6)^. . 
Such students as these^ find themselves becoihing that portion of our edu- 
cational system which is not "prepared to meet the challenges which will 
face them once they leave thQ protective walls of the school'' (Jenkins, 
1971, p. 8). They ate the students who will eventually '^tun'e out" be- 
qause o£ b^^ing "turned off by curriculums which they find to be "dull 
and irrelevant." The> sum total of such educational outcomes will eventually 
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be evidenc.ed in the increasing number o£ students who will someday drop 
out pr graduate from formal education without having acquired any salable 
skill to meet growing needa of the job market or without having organized 
. ahy plans for further. trainin^g Oleyer,. 1972 a, p* 31; Holstein^ 1971, p. ^ 
6; Richardson, 1972, p- 1; Korizek, 1971, p^ 21; ^Goldhammer and Taylor, 
1972, p.. 3). Unfortunately, these ajre the students who will laltimately 
account for the Nation's ever increasing unemployment, welfare, and crime 
rates that, were' alluded to earlier* Rhodes (1969, p* 8) offers some con- 
clusive remarks that seem to sunparize much of what has been presented 
thus far: . , 

"/Sinually,* our educational system turns out millions of* un- 
skilled and untrained graduates and dropouts into a wT)rk force- 
that has no place for than. For mos.t of our youth, the sec- ' , 
ondary school is their last 'chance for full-time education; 
cpnsequently, their preparation fo/ a job^must come during ^ 
high school. Education must be made relevant, idth deqp con- 
cern from the total- student body; it must prepare each youth 
to graduate with a diploma in one 'hand and a job in the other. 
The alternative is clear, we either provide him^with a job or 
fight him in the- street*" ' ' . * » 

Wh^t are^ the possible causes for existing educational and social ills? 
Hoyt (6t al, 1972, pp. 182 and 183) offers tj>e following conclusive remarks 
In addition to an over-emphasis on college preparation, "Much of the cur- 
rent societal unrest exists because of a lack of emphasis on career edu- 
cation for all* It i^s time that career education ^needs of all our 

citizens be recognized and provided for*" - 
** 

Need for Educational ,Change 

"Somehow, and in some vray, school must be made to make senSe to stu- 
derits. . ." ('Bottoms, 1972, p. 3). The logical change that is needed, 
according to Hoyt (ei: al, 1972, p. 30) and Bottoms (et al,. 1972, .p. 3), 
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is that the insufficient emphasis on the educational goal of preparing 
for making, a living be restored by re-integrating the objective of suc- 
cessful preparation for work into curriculum^. 

Hoyt (et al, 1972, pp. 3 and 7) maintains that up to the present, 
"... our schools have seened to operate as though only vocational educa- 
tion personnel are charged with responsibility for helping students ready 
themselves for en5)loyment. JThis of course, is patently ridiculous. . 
He further assert's that teachers can make all types of curriculum rele- 
vant to student vocational interest: 

. *'In these times of increasingly rapid social and .tecbiological 
change, there is no more basically important way of helping stu- 
dents become ready to work than tlifough" the sound acquisition of 
the basic skills of reading, mathanatics, comimmication, the ' , 
social sciences, and' the natural sciences. • % Career implica- 
tions' are in every learning experience, from pre-school to grad- 
uate school ^d beyond." 

As earlier implied, in order to be congruent to the diverse needs 
of all students, curriculum content will have? to be pertinent to all 
varieties of life styles and levels ol career preparation that exist. 
This is not suggesting that the educational goal of preparation for making 
a living ' replace the goal of preparation for living. Rather^ it is a 
conviction that equilibrium among the two purposes of education should 
and can be restored as a viable solution for coping with the .educational 
and social ills that now exist. , ^ / ■ 

As many sources will verify^, iiot all problems that confront society 
are solely caused by education, nor can they be. completely overcome by 
the school systems alone; by the samej token, however, there seems to be 
widespread agreement that education can be a much mqre strategic force 
than it has been in implementing possible solutions (NAM, 1971, p^ 12; 
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Richardson^ 1972, p. 1; Hoyt, et al, 1972, pp, 18 and 26; Peterson, et al 

1971, p. 3; GoldhajMier and Taylor, 1972, p. 4), 

'The 50,000,000 American fairtilies for whom work is necessary, 
but college ijiappropriate, demand alternatives that will offer 
them status, security, satisfaction, and success. They should 
not v;ait much longer" (Hoyt, et al, 1972, p. 29). 

In hearty response to the societal demands for increased responsi- 
bility in contributing a greater share of educational improvements for 
these 50,000,000 American families, much of education's attempts have 
been nationally exerted in the direction of a popular new curriculum 
emphasis called, ''career education." 

Defining the Nature and Scope of Career Education ■ Six Approaches 

Earlier in the introductory paragraph of the rationale, it v;as men- 
tioned that t|ae orientation of an evaluation design is highly dependent 
troon the particular type of program being assessed. This suggests that 
in order for an evaluation design to be effective,, it must be custom 
tailored with close consideration given to the particular nature of the 
program it is to assess. Therefore, before proposing any evaluation de- 
sign, it TOuld first seem essential that the specific nature and scope 
of ca^^er education be defined rathey . explicitly . • > . 

Ever since Commissiondf isidney P. Marland introduced the emphasis of 
career education as a major goal of the U. S. Office of Education, there 
has been deliberate refusal on his part to explicitly define, in specific 
terms, exacts what he had meant by the term "career education" (Bottoms, 
et al, 1972, p. ^5). "In fact, the strategy is to leave the matter of 
definition open to as much dialogue and interaction as possible" (Gold- 
hammer arid Taylor, 1972^ ipp.' 5 and 6) . The assumption of this strategy 
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pre-supposes that leaving the specifics o£ the definition open to inter- 
pretation will inevitably bring rise to healthy controversy and debate 
which, in turn, mil lead to creative development of the concept by edu- 
cational leaders. Consequently, *'the most accurate and honest statement 
to be made at this time is that career education remains to be precisely 
defined" (Goldhammer and Taylor, 1972, p» 5). The general consensus for 
many authorities then, follows that not only will the task of operationally 
defining tHe term career education remain a difficult venture, but also 
that the career education concept itself mil continue to exist as a move- 
ment guided by a non-universally accepted definition for quite sometime 
to come (Miller, 1972 a, p. 16; Hoyt, et al^ 1972, p* 1; Bottoms, et al, 
1972, p. 5; Goldhammer and Taylor, 1972, p. 5). . / j . 

SWanson (1972 a, p. 3) sums it up this way: 

"Career education is almost impossjible to define and its imple- 
mentation is subject to many interpretations and numerous com- 
promises. It has been subjected to endless combinations of ^ 
pressures and preferences to reflect the alternative views of 
t its advocates and its sideline observers." 

As an added insight to the present non -universal definitional state 
of career education. Bottoms (et al, 1972, p. 5) adds, in a somewhat 
humorous vein, the following "words of wisdom": "After all, why should 
'career education* be different frdm any other educational concept?" 

In an attempt to arrive at the specific 'riature and scope of career 
education, and yet accOtot for the variations bf interpretation and em- 
phasis that exist among different definers and implementers of the con- 
cept, the researches has elected to take the advice of Swanson (197? a, 
p. Ill) who states that/TJo single approach to the description or defini- 
tion of career education is complete or adequate." What he feels is 
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needed^ in order to properly define career education is a number of ap- 
preaches which are "mutually reinforcing/' As .a result, I have selected 
to use the types of approaches that were suggested by Swanson C1972 a, 
pp. 109-llJ) and added one of my own (i.e., the concise authoritative ^ 
definitions approach) in hopes that a valid and yet currently representa- 
tive description of career education could be attained. - It should, per- 
haps, be enphasized at this point that these approaches are not mutually 
exclusive, i.e., some overlap inevitably exists, however, as Swanson 
earlier preferred, these approaches will have the end effect of being 
mutually reinforcing in the direction of a specific description of ca- 
reer education. The approaches will be presented in the following order: 
(1) concise authoritative definitions, C2) essential features and compo- 
nents, C3) goals, elements^ and outcomes, (4) educational levels, (5) ed- 
ucational delivery systems, and (6) philosophical assumptions and commit- 
ments . 

Concise Authoritative Definitions . The concise authoritative definitions 
approach is an effort to define the nature and scope of career education 
by offering exan^les of definitions by authorities who have attempted to 
conveniently define career education iin one or two tidy and concise para- 
graphs. 'Six examples of such attempts were selected in order to illus- 
trate and account for the fact that for every definer of career education, 

' I 

there exists a somewhat differing definition for it, e.g., Hoyt^s definir 
tion eirphasizes the career aspect of career education while Evan's defini- 
tion o£ the same concept emphasises the educational aspect of the approach 
CHoyt, et al, 1972, p. 1). The six selected examples are as follows: 

I 

t 
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To Hoyt, "career education is defined as the total effort 
of public education and the concramity aimed at helping all 
individuals to become "familiar with the values of a ivork- 
oriented society, to integrate these values into their 
lives in such a way that work becomes ^possible, meaningful, 
and satisfying to each individual" (Hoyt, et al, 1972, p* 1). 

To Evans, "career education is the total effort of the com- 
munity to develop a personally satisfying succession of 
opportunities for searvice through work, paid or unpaid, ex- 
tending throughout life'' (Hoyt, et al, 1972, p- 1)- 



To Meyer (career education coordinator, V/isconsin DPI; 1972 
b, mimeographed handout) , "Career education is a program to 
facilitate the lifelong process of career development • It 
is a motivational approach to education for people of all 
ages, at all grade levels, and in all subject areas* It is 
designed to maximize career development by^ providing guidance 
and counseling services, by providing experience^Jfco in- 
crease self -awareness by relating learning to joSdipation, ^ 
by utilizing conmmity resources, by blending academic and 
vocational education, by providing exploratoiy wrk exper- 
ience, and by preparing for and placing them in an appro- 
priate 'next step' in their career development, each tine 
they exit from formal education:" 

To Mar land QJ.S* Conmissioner of Education; 1971 e, inter- 
view),/'* . . the term 'career education'* • • is basically 
a point of view, a concept - a concept that says three things: 
First, that career education will be part of the curriculums 
for all students, not just some.. Second, that it will con- 
tinue throughout a youngster's stay in school, from first 
grade through senior high and beyond, if he so selects* And 
third, that every student leaving school will possess the 
skills necessary to give him a start in making a livelihood 
for himself and his family, even if he leaves ^^,bef ore com- 
pleting high school." / \ ^ 

To Pierce (Deputy Commissioner for Occupational and Adult 
Education, U.S.O.E. 1973, p.^-Sl), ''Career education is . . . 
an attitude, an educational philosophy > if you will, that 
must, to be totally effective, pemeate the thinking qf 
every teacher, counselor ^ administrator, board member, and 
parent of this country* 

It is simply a commitment to do everything possible, begin- 
ning with pre-school programs, and continuing through grad- 
uate school to see to it that the educational system prepares 
all children, youth, and adults, who do not suffer from an 
iusuimountable physical, mental or emotional impaimtent, to 
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function at the maxmna of'their ability when they enter 
the lapOT market. ♦ . . Career education, therefore, per- 
vades all of education, and everything ve do is done, not ' 
for the sake of education, but for the sake of the student's 
career preparation/' ' 

6. To Gcldhamer and Taylor (1972, p. 6), "Career education 
considers cuririculum to be systematic - an integrated and 
cumulative series of experiences designed to help each stu- 
dent achieve: (1) increased power to make relevant decisions 
about his life, and (2) increased skill in the performance * 
of his life roles. 

Specifically, career education is designed to capacitate 
individuals for their several lafe roles: economic, com- 
munity, home, avocational, religious and aesthetic. It 
recognizes the centrality of careers in shaping our lives 
by determining or limiting .ivhere we work, where we live, 
our associates, and other dimensions that are significant 
in defining our life style. Designed fof all students, ca- 
reer education should be viewed as lifelong aiTd pervasive, 
permeating the entire school program and ^even extending 
beyond it." > 

In utilizing an eclectic approach, the researcher has developed a 
concise integrated definition of the nature and scope of career educa- 
tion. This definition reflects selicted bits and pieces of the defini- 
tions just mentioned. Like other definitions Of career education, this 
one is also subject to the personal interpretive reasoning of its author 
and is, by no means, offered as a universally accepted definition. 

, Career education is the lifelong pervasive process which systemat- 
ically and sequentially 'provides, through tiie total combined efforts and 
resources of the system of public education, the community, and the indi- 
vidual student, the follovdng: 

4 

\ 

1. A cumulative and dynamic series of experiences offering flex- 
ible insight into the current nature and complexities of our 

\ work-oriented society (e.g., its values. Heeds, opportunities, 
problems and promise) for those persons who must function 
within that society* ' 

* 

2. A suecession of opportunities to relate learning to all the 
variety of educational opportunities, career options and /* 
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consequent life styles that are available - a blending of 
. the academic and theoretical \\vrld of the classroom with 
exposure to the realities of the world of worlCi 

3* Ixtucational assistance required fpr developing the career 
planning and decision-making skills and the personal clar- 
ifications (obtained through self-awareness) that are essen- 
tial for defining and determining life styles • 

It is the ultimate purpose of career education to provide these ^ 

opportunities in a functional way that will facilitate all individuals, 

of all ages, to identify, develop, and perform personally relevant life 

roles (e,g., home, community, religious, work, financial) in the capacity 

they choose to define such roles. Through objective acknowledgement and 

acceptance of a greater variety of li^e styles and alternatives for ca- 

reer preparation, the process of career education is designed to enable 

a greater variety of people to live a more meaningful and' satisfying life 

vdthin the positive contexts of both an individual and societal frame of 

reference* 

Essential Features and* Components . This second approach helps to pro- 
vide some specifics for the generalities that were offered in the con- 
cise authoritative definitions approach. The following list represents 
Mi integrated composite of features and con^xDnents that are considered 
by a variety of authorities to be essential in the proper functioning 
of career*education programs (Swanson, 1972 b, pp. 4 and 5 and 1972 a, 
p. 110; Miller, 1572 b, speech; Hoyt, et al, 1972, pp. 5-10; Keller, 
1972, pp. 3 and 12; KJcIntyre, 1972, pp. 1-4; Goldhammer and Taylor, 1972, 
pp. 5 and 6; lUchardson, 1972, pp. 3-7; Arizona State Advisory Board for 
Vocational Education, 1972, p. 12): 

1. Career education is for all individuals of all ages; it is 
not an exclusive approach just for non-college bound youth. 
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2. Career education is an integral part 'of the total school 
curriculum sequentially involving all grade levBis and all 
course subjects beginning in early childhood and continuing 
throughout life; it is not a discreet entity to be added to 
present curriculum as a separate independent* block, nor can 
it be obtained merely through the establishment of a single 
area vocational center^ 

3. Career education requires utilization of the resources, as 
veil as the active cooperation and i)articipation of all school 
and community members - including tKe student; career educa- 
tion imites students, parenis^ schools, and coinmunity enter- 
prises in a conmon cause. 

4. Career education utilized the motivating force of career 
interest in ^relating the iiiportance and contribution of 
instructional curriculum to the real life career implica- 
tions-^f the world of work. 

5. Career education exposes individuals to the^wide range of 
educational land occupational opportunities that are avail- 
able and orients them to the nature of the society in which 
they must someday function in some chosen capacity; famil- 
iarity with the world of wx>rk creates insight into the work 
attitudes, opportimities, promise, and problems that are a 

* reality in our society; ■ 

' 6, Career education focuses on the development o£ rational 

decision-making skills in order that individuals may; choose 
wisely among the various educational and vocational options , 
that are available to them. 

1 . Career education facilitates the development of self-av/areness 
and clarifies the relationship between different types of work 
and their resulting life styles; this assumes that greater self- 
understanding will lead to the satisfactory selection o£ an in- 
dividually desired style of life and the identification of an 
appropriate career which compliments that life style. 

8. Career education does not force or limit students at an early 
age to make particular decisions pertaining to the select ion 
of a specific. vocation or life style; career education is a 
sequential process that encourages and enables students to 
explore alternative careers^ and life styles upon which tenta- 
tive decisions can be made and modified; career education 
acknowledges the fact that career development is ia life -long 
process and may invol^^e several successive occupational choice§. 

9. Career educatioii enriches traditional school disciplines by 
extending beyond the sterile boundaries of the classroom intb 
the realistic realms^ of the world of wrk; career education 

< ■ . . 
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permits individuals to exit school and^ utilize the }iome bii6> 
the community environments as sources for observation,, wrk 
study, and work experience in selected occupations;** after 
such experiences, the individual may elect to return to*school\, 
for further formal education" (assiiming the pe/son's freedan . 
* to return to foxmal school education is governed mthin the 
limitations of compulsoiy education) . 

** * 
10. Career education provides vocational skill cOTipetency devel; 
opment up to at lea^t job entry levels in selected occupatibns 
that are flexible v/ith the changing needs^ of - a work^oriented 
society^ these minijnum skills are considelred necessary prior 
to any lengthy or final exit from formal education in order' 
to permit a successful transition from< school to. work. 

11.. Career education feature's 100 percent placement, whether 
the student is placed in a full-time 'career of his. choice, 

. . or-^is just temporarily placed in an educational or training 
program which is preparatory in nature; student follow-up 
with successive placements and re-education, if desired, .pro- 
.vides a source for career education's accountability. 



/ 



12. Career education i^ neither ''anti-college" nor "pro-tr^de ' 
skills;" it provides equal en?)hasis and acceptance for all 
dimensions of the "'occupation^' hierarchy^' including the 
various modes of career preparation and the differingflife 
styles which accommodate than; such objectivity fully pro- 
vides the security and protection of individual freedom of ^ 
- choice. 

1Z> Career education is not synonymous \yith the term vocational 
or technical education, hov;evet, vo.cational -education is an 
essential component of career education; ^the same token, 
* career education is not in itself occupational straining, * 
even though occupational training as a necessary ingredfent 
of career education. *' ^ ' 

» • ' •* " 

14. .Career education in no way conflicts with other educational 
objectives; it is rather an approach to restore equilibrium 
among the polar objectives of education: preparation for 
livihg vs preparation for making a living. 

Go^ls, Elements and Outcomes . ' This third approach describes career edu- 
cation in terms of defeired pre-determined performance outcome? and. char- 
acteristics that individuals should cumulatively acquire after exposure 
to program treatment, through detailed examination of^a variety of car 
reer education handbooks, guides and instructional units, Keller /C1972, 
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pp. 16-27) has developed a synopsis o£ selected"* career education goals 
for schools. This .synopsis may be referred to in* Appendix A of this , 
report* 

The Ai'izona State Advisory Board for Vocational Education (1972, p» 

10) summarizes the goals of career education into the folloiving three 

« 

general goals: 

^ Learning to Live: This means developing a self -awareness of 
one's capabilities and developing the ability to deal with 
leisure time and society in general. 

Learning to Learn: This involvjss motivating students so that 
they want to learn the basic educational subjects. This can 
be done by making the subjects meaningful and by relating them 
to^ the real world of, work. 

Learning to Make a Living: This means preparing students so 
that they have the capability to support themselves economically 
and to become productive members of the coimtunity. 

After acknowledging the fact that "there may be many ^^ts of ele- 
ments which identify or describe career education," the staff at Ohio 
State's Center for Vocational and Technical Education have identified 
eight elements along with eight respective desired outcomes for use in 
their Comprehensive Career Education Nfodel (CCEM) project (Miller, 1972 
pp. 20-23). These, elements and outcomes may be referred to in Appendix 
B of this report. . - , ' ^ 

Educational Levels. The ^fourth appr^^cli to defining career education 
focuses on the sequential and ajanul|Ltive process of iinplementing career 
education along educational levels? starting with early childhood and 
proceeding through adult and cbntinuing education. Each level of this 
cumulative process has specific objectives that are .geared toward phe 
leveFof comprehension that can be expected of the individuals for their 
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respective grade level. In reviewing a variety of proposed nx)dels which 
utilized educational levels in defining school-based career education, it 
was discovered that a number of features seemed to be coninon to them all. 
Figure II (p. 1-28) illustrates these coWon features in a single chart 
which is based on an integration of the features and key terms of several 
inodels (Keller, 1972, pp. 9-13; Holstein, 1971, p. 6; Arizona State M- 
• , visory Board for Vocational Education, 1972, p. ^11; Mclntyre, 1972, p. 2; 
Richardson, 1972, p. 5; Bailey, 1971, pp. 15-27). -The foiroat for the * 
chart is patterned after Figure I (Continuum Depicting a Lifetime, p. 
I-ll) which was developed earlier in the rationale. 

In order to give an example of some of the sequential and cumulative 

« 

processes that occur along the various educational levels of the continuum 
illustrated in Figure I J (p. 1-28), a chart depicting a "career selection 
pattern" developed by Goldhammer and Taylor (1972, p. 139) is presented 
in Figure III (p. 1-28). 

The Wisconsin Career Development Scope and Sequence Model (Drier, 
1972 b, p. 13). along with its career development concepts, as shown in 
^ Figure IV (p. 1-29), is another useful example which illustrates the 
educational levels approach. This state guide depicts, not only the 
sequential order in whiclv the various career development concepts could 
be implemented, but also indicates the relative emphasis that each of 
the concepts might be stressed. 

Educational Delivery Systms . The delivery systems approach to defining 
career education focuses in differing amounts (depending on the differing 
\ peculiarities ^pf various programs), upon the resources that are available 

to the educative function (including such things as student time and effort, 

/ 
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Figure II. 

'Continuum Describing the Process .o£ 
Career Education According, to Educational Levels 
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Figure III 
Career Selection Pattern 



Elementary 

Exploration of what the 
world and the society in 
which one lives has to 
offer 

Learning basic skills and 
what individuals do to 
become workers in 
various fields 



Middle Grades & Jr. High * 

Exploration of various career ' 
possibilities consistent with, 
the p o t e n tialities of the 
individual child — actual in* 
voJvement in activities 
performed in certain careers 
and learning some basic sicills 
needed for success in these 
areas. Studying the stylc% of 
life and conditions und.^r 
which people engaged in 
vinous careers live and worlc 



Senior HigH ^ 

Narrowing the choice of a 
^career to a few limited possi- 
bilities. Studying background 
knowledge and acquiring^ 
background and fundamental 
skills. Narrowing field of 
choice to increasingly 'fewer 
options. Learning more about 
self and training requirements 
for selected career areas. ' 



^ Post High 
Basic 

occupational 
or skilled 
or professional 
preparations - 
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' ' Figure IV 

Wisconsin- Career Development Scope 
and Sequence Model 
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Adolescence 
7-9 
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* Career Envelopment Concepts 

1, An under st:&nding and acceptance of self is important throughout life, 

2, Persons need to" be recognized as haying dignity and worth, 

3, Occupations exist for a purpose^ ^ " ^ " 

4, There is a wide variety of careers,, which may be classified in several ways. 

5, Work means di-fferent things to different people. 

6, Education and work are interrelated. , ' 

7, Individuals differ in* their interests, abilities, attitudes and values. 
81. Occupational supply and demand' has an impact on career planning. 

9. Job specialization creates interdependency. 

10. Environment andl individual potential interact to influence career development. 

,11^ Occupations and life styles are interrelated. - ^ 

12. Individuals can learn to perform adequately in a variety of occupations.. 

13. Career development requires a continuous and sequential series of choices.^ 

14. Various groups ar^d institutions influence the Mature and structure of work. ^ 

15. Individuals are responsible for their career planning. 

16. Job characteristics and individuals must be flexible in a changing society. 
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facult)^ time and effort > curriculum, materials. and equipment, aiid space) • 

It also specifies how these resources might be implemented in the direc- 

tioji o£-program goals* As Swanspn (1972 a, p. 110) states; ''The deliwry 

system for career^ education is the motion elemei\t of the program as viewed 

''in its day-to-day operation." . • 

ITie CCBt staff at Ohio State Q^liller, 1972 a, pp. 25* and 26) have 

developed *'both a standardized fjpmat and guidelines for curriculum unit 

revision or development'' for use in .designing delivery §ystans for their 

career education project. The guidelines'and. format are as follows: 

* 1^ A teacher *s guide which^ specif ies : , . 

k. The rationale for the unit. - , ' 

B. Intended use of the unit by suggested grade level, subject 
area(s), time, grouping, and special' considerations, 

G. Gqals and performance objectives. - 

2.. Teaching procedures r ^ • * 

" ' * A. Learning ^ctivit^ies . ' ^ 

B; Resources.-*. - , . 

'C. Performance Evaluation. ' ^ ^' 

i 

3. Teaching materials . 

4. Evaliiatior procedures. 

» » 

5. Specifications for in-serVice training of the teacher or 
implementing. , the unit. . , 

Another example; which illustrates how resources might be implemented 

into action is offered by Hoyt (et al, 1972, p.. ISO). His delivery sys- 

tern which includes ten action steps for implementing careey education are 

X 

shown in Table I as follows: 

* 
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Table I 



Ten Action Steps for Implementing Career Education 



I* m 



Phage I • 

!♦ Organize the appropriate interactive network of interested, indivi- 
duals and groups* 

Gain arTunderstanding of the concepts of career education and , 
establisii appropriate educational objectives. . 

Phase II. , " ' 

3. ^ Study the current educational system to determine the changes 
. .necessary to turn it into a true career education system* 

4. Inventory' and marshal all available resources. 

5. Begin planning the career education system most appropriate 
for your community. 

Phase III 

'6. Seek the cooperation of all necessary organizations, in- 
stitutions, and individuals. 

7. Iirplement the system (note p. Ill -15). 

8. Put the evalliative process in operation to determine how 
well the system is workings 

' Phase IV ' ' 

9/ Create a feedback system to use .evaluation findings 
, to adapt and improve career education programs. 

10. Make provision for *i program of maintenance to sus- 
. tain the vital part^ of the system and the initiative, 
and tie these activities into the interactive network. 



Philosophical' Assumptions and Commitments 



Premises and Propositions Underlying Careei* Education , ^n order to 
better understand the total nature and scope of career education, the 
philosophical assumptions and commitments from which it has emerged must 
be considered. Nfost conceptual de£initi9ns of career education are actually 
a product of a wofk-oriented society^ 3134, therefore, represents some pretty 
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strong convictions pertaining to the' ethical nature of people, society, 
and economics. The one .basic premise which seems to underlie the whole 
concept o£ career education, as stated by Hoyt (et al, 1972, p. 1 and 2), 
is: 

" . "that the society is and should be achievejnent-oriented, with the 
development of the individual as its primary objective, but with 
the development best accomplished and measured through service to 
others and to the whole. That conviction allows for, and, in fact, 
demands, that man be viewed as more than a working machine — that 
he is what he believes, what he treasures.. But above all, it con- 
• ' eludes that the best measure o'f what he is, is what he achieves in 
the development of his talents and in his service to hijnself and 
to his fellow inanV 

That conviction gives primacy to the employment role because 
the average adult male spends more time in pursuit of his working 
career than ?ny other single activity. . 

In elaboration of this basic premise, Hoyt was careful in emphasizing j 
that "achievement is not limited to the labor market,^' but rather extends 
into all sectors of an individual's life, including his personal leisure 
time; "Therefore, career education is preparation of all niianingful and 
productive activity. . 

The following four propositions by Hoyt (et al, 1972/pp. 66 and 67) 
are offered at this point in order to provide an idea of /some of the 
assumptions upon which definitions of career education Sieem commonly based: 

1. Given ^ual natural resources and ingenuity among its ^feople, 

the expected productivity per unit of population^asscJcia » 
with any society bears a direct relationship to, that Nation's <S 
cpmmitinent to the TOrk ethic . > y C , 

2. The classical version of the' Protestant work ethic in^Anerican 
society^ is currently being eroded by a variety of ^forces and 
is no longer a viable concept for many members of our society. 

3. No gr'^at civilization in history has continued to prosper 
after it abandoned its basic coiranitment to a work ethic (as 
Bottoms, et al, 1972, p. 2, explains within the same context: 
"This was true of such nations as Persia, Egypt, Greece, Rome, 
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- Spain, •Portugal, France and England")- There is no reason to 
believe the United Stat^s is exempt from the historical pattern* 

4. the basic goal o£ career education is the restoration o£ various 
forms o£ the wrk^thic adapted to reflect liew social and eco- 
nomic realities as a strong and viable force throughout our 
society. 

To further clarify just exactly what might be referred to as the 
"classical work ethics," Hoyt (et al, 1972, p, 67) provides the following 
exan^les:^ 

1. -All honest work possesses ijinate dignity and worth • 

2^ Excellence can be attained and is rewarded in any occupation. 

3. ; One should strive to do his best in whatever ivork he does. 

4. The worker who is satisfied with doing less than his best is, 
to some extent, dissatisfied with himself. 

5. The x:ontributions one can make to society stem, to a large 
extent, from the work one does. 

* / 

6. Work is seen as possessing personal, as well as financial re- 
wards for the worker and the phrase, "A task well done is its 
own rerard" has real meaning, 

7. Persons to wiiom the work ethic is meaningfiil ivant to work, 
preparte themselves for work and actively seek to wrk; they 
are, most of the time happier wheji they are working than 
when they are not. 

8. A significant portion of the pride individuals have in them- 
selves is found through accomplishment of their work. 

9. Hard work is seen as the best and surest route to the highest 
level of occupational success possible for the individual 

The various concepts pertaining to the classic* v\?ork ethic are 

applicable to some individuls while others strongly oppose them^'CHoyt, 

et al, 1972, p. 69). There seems to be a new breed of worker entering 

the labor market that does not completely agree on all the aspects of 

the classical work ethics^ e.g. , this new breed of wrker appears to be 

less satisfied by pay, and more interested in the meaningfulness of work 

J 
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tHoyt, et al, 1972, pp^ 23, 24 and 40; Keller, 1972, p^ 5) • Our educa- 
tional system needs to account for l^e variety o£ differing amounts of 
acceptance that now exists among different persons for the conceptual 
assumptions of the classical TOrk ethics. Career education is dedicated 
to this cause in its efforts to help individuals acquire a truly personal 
meaning of vork that is uniquely valid for them . 

*'Viliere the v;ork ethic can be used as a basis for such meaning, 
it should be fully utilized. ]\Tiere the work ethic, in its 
classic form, cannot be applied, then alternative ways of bririg- 
' ing iiieaning to work must be developed and used'* (Hoyt, et al, 
1972,, p\ 69). , 

Controversy Concerning the Teaching of Values and Work Ethics in 
School. This brings the rationale to the issue which presently appears 
'to be the most controversial aspect of the career education concept 
teaching values in public education with the major source of criticism 
being the consideration of the work ethic and the anphasis it places on 
the quality of work, and the "good" worker characteristics (Hoyt, et al, 
1972, p. 39). One exainple which seems to illustrate this controversial 
aspect of career education quite well was written in a recent article of , 
the Phi Delta Kappan by Nash and Agne (1973, pp. 372-377). In this art- 
icle, they made an analysis of the ideological premises underlying career 
education proposals and indicated that there are many positive and humane 
aspects of career education while at the same time "a number of key assump- 
tions have been left unexamined," e.g., according to them, such assumptions 

as huinan behavior is achievement -motivated and the continued existence of 

f. 

a corporate social otder which values the classical vork ethics as being^ 
"good" have been left unchallenged. In general, this group felt that 
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"because of the excessive claims and the absence of significant , self - 
criticism, we believe that the career education prospectives must undejgo 
careful > constructive examination/* 

» Perhaps in some cases the perspectives of career education have been 
left unexamined. By the same token, however, much thought has been in- 
vested toward consideration of what might happen if the eroding work ethic 
weye left unattended to. For example, authorities (Bottoms, et al, 1972, 
p, 2; Keller, 1972, p. 5; Goldhammer, 1972, p, 4) have pointed out that 
the erosion of the work ethic over the last thirty years has now come 
to the point where the lAiited State's position of leadership in the world 
has been jeopardized. As Bottoms Get al, 1972, p, 2) asserts: 

"The relative loss of the work ethic in our country, coupled 
with the relative gain of the work ethic in such countries as 
Russia and the Peoples Republic of China, is seen[, by many 
people, as a basic reason \kiy such countries appear to be - 
catching up with the United States as leading world powers. 
It is a very serious thing indeed." 

The goals of education, whatever they may be, are a product of the 
society from which they evolved. Therefore, they tend to reflect very . 
closely the values and influences of the prevailing cultural ^and eco- 
nomic factors of that society. The concept of work, along with the pre- 
vailing values and ethics which accommodate it, are a reality of our 
society; the average adult male spends most of his waking hours in an 
activity called "work." ". . . if work is a reality of our society, then 
should it not be an integral part of our total educational system?" (Keller, 
1972, p. 1). 
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Skepticism' Concerning the Originality of Career Education , Due to 
growing skepticism concerning sloganizing, career education has been 
knavn to be attacked by critics who question its originality* Actually, 
"career education concepts are not new," they "have been finder experiment 
and^ development for many years" (Jloyt, et al, 1972, p. 13). No one has 
recognized this fact more than Commissioner l^rland who, among many others, 
made contributions to the concept during Ms early professional life as 
a local school superintendent. Therefore, even though.it may have been 
the policy of various types of administrations in the past to claim 
originality of newly adopted .policies, it doeS^ot seem to be the policy, 
of the Commissioner's stand on career education. As Hoyt (et"-al, 1972, ^ 
p. 13) states, "No one appears more conscious than Commis^ionp Mar land 
of the need to enphasize the continuity of career education's development." 
Therefore, as long as the precursors of career education have not attempted 
*6 claim originality of the concept, it seems only reasonable that the 
issue of originality not be considered a legitimate source of criticism. 
What can be recognized as being c^iginal about the career education con- 
cept, ha^ever, is the relatively new emphasis and coherence that it has, 
attained because. of nation-wide support.^ 

Support for Career Education as a Possible Solution 

"Nothing in the world is so powerful as an idea whose time has come" 
(Victor Hugo) . 

"Among the reasons for believing that career education is a con- 
cept whose time hasj come are that; (a) it'^Ras emerged at a moment 
when dissatisfaction with educational practices and outcomes are 
^ at a peak, and (b) it promises to attack and improve some of the 
apparent sources of that dissatisfaction" CHoyt, et al, 1972, p. 17). 

46 
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Throughout the nation, civic, industrial and legislative groups at 

the local, state and federal levels have given wholehearted endorsement 

\yand support to career education (Swanson, 1972 a, p. 107; Gbldhamner and 

Taylor, 1972, pp/4 and 5; Marland, 1972 c, p^ 10). 'A nationwide field 

-I 

survey of past and present career education efforts conducted by Palo 
Alto Educational Systems, Inc. (1972,, p. 60) reveals as a "single most 
significant conclusion. . . that there is an unusually high level of 
interest in career education within public schools and. state depart- 
ments of education throughout the country." 

In his State of the Union message, January 20, 1972, foimer^ President 
Nixon stated that career education would be an area of major eii5)hasis. He 
also said that schools need to direct more effort in^ providing people with 
broader exposure to and preparation for gainful employment in the world of 
work. In an effort to give more oiqjhasis to career education, he revealed 
intentions of pro|>osiftg intensified support for develpping and testing 

career education models. (Keller, 1972, foreword) 

'> 

The following comments by Hoyt (et al, 1972, foreword) further indi- 

'cate that career education is an idea whose time has come: 

"Career education is *in* in American education this year. The 
Natidnal Education Association ^ has endorsed it and also the 
National Association of CMef State School Officers, the American 
Vocational Association, and the National Advisory Council on Vo- 
cational Education. Tne Associatfbn of Secondary Sctool Prin- 
cipals has acclaimed it, and the concept is attractive to the 
National Congress of Parent -Teacher Association. The American 
Association of School Administrators is holding cuiiferences on 
careet education through its National Academy of School Executives. 
The American Associatirai of Junior Colleges has joined the favor- * 
able chorus. Outside education, the response fvm organized labor, 
the U. S. Chajnber of Commerce and similar groups have beep equally 
^ approving. Nowhere does one find organized opposition." 



Mich of the national acclaim -for career education can be attributed 
to the effort? of the Coanmissioner of the U. S. Office of Education (USOE) > 
Sidney P. ^farland, Jr. , who "has made career education the password of his 
admijii St ration"; ^^arland has not only verbally promoted career education, 
but has also "allocated iiuch of the discretionary m3ne}c available to him 
through various educational appropriations to inplementing the concept" 
(Hoyt, et al, 1972, fore;>^ard). The following 'comments by the ^iranissioner 
(^^arland, 1971 d, speech) exeirplif ies the kind of «bacldjig he has been 
giving the career education approach ever since its first conception: 

JL 

^ "all education is career education, or should be. And all our 
efforts as educators must be bent on preparing students ei-ther 
to become properly, usefully eirployed immediately upon gradua- > 
tion from high school or to go on to further formal education. 
Anything else^ is dangerous nonsense."- ^ « , 

The follCT^ring comments by >iargules (Tuckman^no date listed, p. 2) 
depicts a rather positive general public reactJLon to fferland*s "commit- 
, merit to a national effort ;in Career Education" and also provides impli- 
cations of increasing emphasis and promise for the^new approach: 

'The wave of enthusiasm \Aich confronted Commissioner Sidney 
Jferland's commitment to a national effort in ^'Career Education^ 
is gaining momentum at all levels of education. The concept has 
caught the imagination of educators across the country who have 
been searching for a viable response to the hue and cry of tax- 
payers, parents and students alike, for a more relevant educa- 
tional process. Despite the fact that career education has been 
sporadically implemented in innovative school districts, these 
efforts *will surely increase as educational leaders begin to 
recognize the far-reaching implications of career development." 

Hie resulting evolution of a national interest in the career educa- 
tion concept is not really too surprising when one considers the fopu- 
lating forces from which it originated, i.e, existing educational ills 
coupled with the additional impetus of federal money earaiarked for this 
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purpose and a promising increase of local, state and federal en^jhasis for 
the approach. Hoyt (et al, 1972, p» 183) brings to point the follo;>ring 
closing reminder: 

k 

''Finally, it is iji^rtarit to note that in aslcing for career edu- 
cation^ we base our requests on the needs of the 83 percent of 
our citizens - both young, and adults, -in school and out of school - 
\fciO will never attain a four-year college degree. It is time 
that/teducation serve best this real majority of our citizens." 

/ t ■ . - 

Summary and Conclusions 

Qirrent educational literature is filled with statements that are: 

(1) exposing existing problems that are now facing education, (2) con- 

vincing^that there is a need for educational* change, and (3) illustrating 

that many significant individuals and groups at the local, state, and 

federal levels nationally support and promote career education as a poss- 

ible solution to the, ills 'of education. Career education has been 

nationally ijiplemented in many innovative sdhools throughout ihe country, 

and, as was earlier documented, this approach seems to be .much more than 

just a passing fad. 

. - / 

Need for Effective Evaluation of Career Education Programs. Each year, 
huge amounts of federal and local tax fimds are being spent on the pur- 
pose of public education. Mich of these fixnds have been earmarked spe- 
cifically for the purpose of iii5)lementing career education. .Both Congress 
and the general public are beginning to wonder how their money is being 
spent.. Local community members are saying: "If we are to entrust to you 
educators our resources and our kids, we want some objective measure of 
whether or not you did J" (Marylan4 State Board of Education, 1972, p. 29) 
In addition, teachers and administrators are anxious to find out how some 
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of the new programs they are involved in are coming along. They need to 
'^eam the strengths and weaknesses o£ the programs and how theSe pro- 
grams might be iirproved" (CVens, 1973, p. 73) a 

Even with the understanding that large siins of money are being spent 
in the educational enterprise, one might still legitimately ask the ques- 
tion: '^Vhy is evaluation itself in^rtant in eciication?" In answer to 
this question, Finn (1973, p. ^11) offers the following five, reasons: 

1. To add to the substantive knowledge of educational processes* 

i, 

' ^2. To provide infoimation in order to adjust, discard or other- 
wise change the application of an ongoing educational process.. 
h ' / ^ 

5. To provide justification for a political-social-economic action 
relating to education. 

-9 

4. To create a productioli (usually paper)^ which can move through 
educational bureaucratic systems and thus keep these systems 

' operative . ' , . ' 

5. To provide instruments which may be used to carry information 
V, on siiccess of the process .to the educational community. 

There seems to be a lot of agreanent among educators on the need for 

educational evaluation. R>r exanple, Hyler (1969, p. 2140) states that 

'"Riis is a period of great potential for progress in American,, Education. 

Such progress can only be achieved if evaluation methods are rigorously 

used in the development pf educational programs." \^SQX^g (1971, p. 199), 

Coster and Morgan (1969, p. 23) similarly agree bn the ir^jortance of 

educatipjal eiJ'aluation in the following ti^o statements: (1) IVysong - 

"Evaluation is ^n ijnportant part raany program beeause it can^ contribute 

to the improvement of the program and can provJ.de information useful in 

communicating program accomplishments and needs to Various publics," and 

(2) Coster and Morgan - •\ . evaluation is not merely essential, but 

absolutely mandatory as a key element in progress and goal realization." 
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Failure of Tr aditional hfethods of Evaluation. Despite the need for eval- 
uation, traditional methods of evaluation for innovative educational pro- 
grams, such as career education, have in the past fa^d in their attempts 
to trui^ assess- program effectiveness (Gaba, 1973, .pp. 1 and 2) . Ever 
since the passage of the Elementary and Secondary Education Act of 1965, 
schools receiving funding under its provisions were required to provide 
an evaluation of program efforts. It was at this time that it "became 
clear that very little was knom about public school evaluation" (Ott, 
1973, p. 112),, and according to Cuba (1973, pp, 1-9), there still seems 
to be much room for .continuing research and im[prQyement in educational 
evaluation: ". . . evaluation as we know it lias failed. . . the world of 
evaluation does indeed require reshaping" (diba, 1973,, p. 2). 

"The American educational establishment is currently making a mass- 
ive effort* at self^inprovement" (Cuba, 1973, p. 1). Consequently, much 
time, effort, and monpy have been expended on innovative programs such as 
career education. However, due to deficiencies in the traditional methods 
of evaluation, these promising programs have been left "unproved" (Cuba, 
1973, pp. 1 and 2). 

Need for Increased Efforts Directed Toward Developing an Evaluation Design 
for Career Education Prograjng .. Since: (1) the importance of improved 
changes in the field' of education' has been ;recognized, (2) career educa- 
tion has been offered as a possible solution, (3) there is ^ promising in- 
crease of local , state and federal enphasis' for the approach, and C4)-tra- 
ditional methods of evaluation for innovative programs such as career 
education have in the past failed, increased efforts need l!o be directed 
toward the development of an evaluation systein to measure the effectiveness^ 
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of school-based career education programs* It is Juteresting to note 

« 

"that the dissatisfactions which produced the demand for .career educa- 

, / ' • I ^ , ^ • . C 

tion were the same as those underlying otfier demanded educational re- 
foims. All involved the demand for accountability" (Hoyt, et al, J972, 
pp. 45 and 46). , ' - ' * 

In an effort to help ^in determining the real utility and pi-omise 
that career education programs have to offer, the evaluation design, 
which follows in this report, was developed. ^ It is hoped that the sug- 
gested design will ultimately assist local agencies in determining the 
effectiveness that their school-based career education program liad in 
contributing to the educational goals they've established. ^--^ 

Definition of Terms and Limitations 

Limitations of the Study 

I. It should be acknowledged that the evaluation design developed in 
this project is an adaptation of the Stufflebeam-Guba. (Stufflebeam, 
et al, 1971) CIPP model. The career education evaluation design 
developed in this report utilizes the four classifications of gval- 
uation (called context, input, process and product) that were ident- 
ified in the CIPP model, in describing the four stages of evaluation 
for career education. Just as the four stages of evaluation are used 
in the CIPP model for yielding information pertaining to different 
' classes. of decision-making, so are they used in the career education 
evaluation design for evaluating the four stages of progiam iiuprove- 
ment for career education that were identified in this project. • 
LI. V/orthen (1973,- pp. 30 31) suggests as a miiiimaX first step in 
4:he development of an "idfeal" educational evaluation model, that 

l * 
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evaluation tasks be structured iif ;^*uxta|)osition against Stufflebeam's 
four kinds of Evaluation iMch' were identified in the preceding 

paragraph (note Figure v, p* 1-440 ♦ It onfe^Hvere to successfully take 

... , • /.7 . - 

orf such an enormous task, the end prpduqt would probably result in 

as near an ''ideal" evaluation design as is humanly possible. Such 

an ideal model would then restilt irf an infinitely complex "taxonomy 

of evaluation designs*" - 

*As Worthen (1973, pp. ^-8 angl 30.) further suggests, however, a 

"* * " • 

project such as this is a most "laborious" and painstaking process, 

"Given the staggering coji^lexity of evaluation design processes, 
. when viewed in their entirety, it would seem that attempting 
to create a taxonomy of ;^Vaiiiation designs TOuld be both, naive 
* and inappropriate at this point ." 



Due to the fact that the evaluation design presented in this re- 
port was conducted within limited time and resource limitations, the 
ideal ^evaluation model just /'scussed can presently be considered be- 
ypnd the mission\of this project. ' \ 

All career education programs are unique in that ^ach school 
system. differs in such aspects as interpretation of educational 
needs, accumulation and availability of resources and methods of 
'iirplementation. In order for an evaluatioij "^design tp be applicable 
to such a wide variety of programs, it must be somewhat standardized. 
The more standardized an evaluation design is expected to be, the 
less detail it can usually afford to stipulate. Th^ evaluation 
sign developed in this pronect is therefore intended to compromise 
the following two major criteria: (1) to be general enough to be/ 
relevant to a variety of different career education programs and 
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Figure V 
J — ^ 

A Proposed Classification Structure For An 
Ideal Educational Evaluation Model* 



EVALUATION TASKS 



TYPES OP E^'ALIJATIO^I 



CONTEXT E\7AL. 



IMPUT EVAL. 



P^.OCESS ESJKL. 



I. Developing a climate 
among educators that is 
supportive of evalu- 
ation . 

A. Assessing educator's; 
existing attitudes 
toward evaluation. 

B. Estjablishing rap- 
porjt with partici- 
pants. 



etq. 

II. Planning and focusing 
an evaluation. 

A. Establishing pre- 
mises which will ^ 
guide the evalu- 
ation . 

B. Determining what is 
to be evaluated and 
in" what . seauence . 



• etc. 

III. Selecting or construc- 
ting instruments. ^ 



etc. 



/ i. . * 

* Adapted fr4m Worthen (1973, p. 31) by placing St;uf flebean' s (tt al. , 197X) four types 
of evaluation in juxtaposition against Owen's (1973, pp. 73-79) suggested tasks and roles 
of evaluation specialists in education (note Appendices E and F for the, remaining roles 
and tasks suggested by Owens and Stuff lebeam) . 
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^ ^et, (2) be specific enough to be functicmally useful, i.e., offer 
\ enough details so that educators •do not have to develop their own 
career education evaluation prograins de novo. 

In suirmary, it may Be asserted that, . . the design and 
analysis of educational, evaluatioh is a inost complex and diffi- 
culf undertaking" (Stuff lebeam, 1973, p. 27); . . the process 
of designing evaluation is much more conplex than it appears to 
be cn the surface" (Worthen, 1973, p. 30). Wljile it is doubtlessly 
rea aized that many sections of the career education evaluation 
* design, developed in this report, could ideally be explicated in 
greater detail and conplexity, it is in the interest of applica- 
bility and practicality that the design is presented in the format 
as will be later illu^trateSTxToiapter III. - ^ 

III. Ironically, the evaluation design illustrated in Chapter III, with 
all of its flow charts, inputs, processes and outputs, almost looks 

too mechanical and impersonal to be describing an approach which is, I 
by nature, as personal as, career education seeks to be. This seems 
especially true when one is forced to label and, therefore,^ picture 
. students and staff as inputs, 'processes and outputs. Despite what- 
ever qjnplied impersonalization may be inherent in the format of the 
evaluation design presented in Chapter III, career education may 
still be distinctly differentiated as a very potential and relevant 
approach for personalizing and indiviaualizing instruction* 

IV. At first glance, the rationale of this report may appear to have over- 
emphasized educational ills to the exten"^ that education is viewed 
as: (1) an institution which has contribued nearly nothing to society 
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and (2) a primary cause of social ills. These two viev^oints, how- 
ever, are neither the expressed intent of the rationale nor are they 
necessarily a valid assunption* Tl* follpwing list of statements by 
Hoyt C1972, p. 17) illustrate whf: 

1. >bst of the dissatisfaction directed toward education is, in 
reality, frustration with social ills for ^>^iich schools bear 
little fault djid to the solution of which they have been able 
to make only modest contributions. 

2. ^feny of American education's problems are the consequences of 
success and its strengths cxitweigh its weakness. 

3. The atmosphere is permeated, with demands for educational re- 
form,, and career education must be viewed in its relationshij) 
to these demands. 

Along ^ this same vein, the ultimate goal of evaluation should not 
be considered that of labeling a program as either being "good" or 
*'bad." Even though at times such labels may result as un-snticipated 
and una\^oidable by-products of evaluation, they should not necessarily 
be considered evaluation's ^rime objective. The following comment by 
Stufflebeam (et al, 1971, cover page) will attest to this: 'The pur- 
pose of evaluation is to^ improve not to prove." 
Objective four for this study was to "Apply the evaluation system to 
a selected career education program and report the re:^ults." Once 
the design for- the evaluation system was completed, it was soon dis- 
covered that a complete and\ comprehensive application of the career 
education evaluation design, including all the necessary phases from 
context evaluation, through product evaluation, would require a long- 
itudinal study involving anywhere from at least one to five years for 
final coirpletion. Therefore, in due consideration of the project's 
time limitations, it was determined by thp, project participants that 

\ ■ ■ ' 

1-46 

" . . 56 ■ 



J 



the application, as stated in objective four, be considered beyond 
the present scope o£ this study. ^ 

As a more practical alternative .to objective four, .the researcher 
(along with Dr. Orville Nelson, Professor and Research and Evaluation 
Specialist, Center for VIAE, W-Stout and Dr. Harold Half in. Professor 
and Director of the Center for VIAE, UlV-Sout) participated in a third 
party career education evaluation that was conducted under the team 
directorship of Dr. Carlyle W. Gilbertson, Professor and Chairman, 
Department of Counseling and Personnel Services, Ul^- Stout. Hie pro- 
gram being assessed by the third party evaluation team was an exemplaiy 
project in vocational education, titled "Articulation of Occupational 
Orientation, Education and Placement in Private and Public Elementary, 
Secondary j^d Post-Secondary Schools.'* The major purpose of the exem- 
plary project was to provide a conprehensive career education to a 
variety of persons ranging in all ages and grade levels. 

Using the experimental and control group approach, the research 
design euployed by the third party evaluation team closely parallels 
the final two stages of this report J s career education .^valuation 
design called process and product evaj-uation. The primary focus of 
the process evaluation was on the ongoing procedures and activities, 
whereas the primary focus of the product evaluation was on the extent 
to vtfiich objectives and outcomes were attained. ^ For further details 
concerning: (1) objectives of the evaluation, (2) evaltiation activ- 
ities, (3) methods and procedures, (4) research design, (5) instru- ' 
ments developed, (6) data collection and analysis procedures, (7) var- 
iables measured, and (8) formats used for this third party evaluation, 
refer to y^pendix M. ^ >y \ 
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Definition of Terms 

Appendix L consists of definitions of vocabulaiy words that are re- 
lated to the fields of evaluation and career education. The process of 
evaluation, which involves such a high degree of infonnation exchange, 
is highly dependent upon a nrutual understanding of terminology used be- 
tVi'een evaluators and decision -makers. The vocabulary list has, .therefore, 
^been developed and included in this report for the following two reasons: 
(1) to provide a basis upon which a mutual understanding of the terms 
used in this report can be interpreted and (2) to offer a somewhat stan- 
dardized basis which evaluators and decision-makers may use as a form of 
reference in communicating meaning. 

^fore of the terns included in Appendix L pertain to the field of 
career education than tliey do to the field of evaluation. The source 
of each tem is credited after each definition within the Appendix, 
For a more complete vocabulary list of terms pertaining to the field ^ 
of evaluation, refer to some of the appropriate sources listed in 
-^endix I. . ■ 

r 
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CHAPTER II 
flEm^ .USED. IN TOE STUDY 

f 

The initial method of study used in: (1) identifying the nature of 
evaluation, (2) developing the evaluation design, and (3) identify4ng the 
tasks and considerations involved in administering it consisted of an 
extensive review of avaU.able literature pertaining to career education 
and evaluation- Sources of review were primarily made from current . 
periodicals, boolcs and ERIC-materials. 

In developing the rough draft of the evaluation design, the researcher 
modified and integrated models and methodologies that were currently being 

'used by evaluation specialists in the field o£ education. These integrate<i 
models and methodologies were synthesized vdth special consideration given 
to their adaptability to the four stages of program improvement for career 
education that were identified by the researcher. Similar methods of 
synthesis were used in compositing variables, data collection and analysis 
methods, feedback procedures and foimats, and criteria for the assessment 

,of the evaluation design. Composites such as these were structured into 
the .portion of the research report concerning tasks and considerations 
involved in developing and administering the evaluation design. Hie two 
synopses of evaluation instnments located in Appendixes C and D were 
extracted and conposited from a variety of major test publishers, and from 
current career education handbooks, guides, and project reports. 

An advisory committee w^s identified at the beginning of this study. 
Consultation with the committee v/as made throughout the entire study until 
the final draft was conpleted. 

59 
• ii-i 



CHAPTER III 
RESULTS 



Definition of Evaluation 

A review of literature has revealed the attenpts of many authorities 

|o* accurately define the term evaluation (Stufflebeam, 1973; Alkin, 1970; 

Coster and ^brgan, 1969; Stufflebeam, et al, 1971; Tyler, 1969; Cutlip, . 

1971; Kincaid, 1968; Vfysmgi 1971; Nelson, 1973; J^iyland State Department 

of Education, 1972; Drier, et al, 1972 a; to mention a few). In combining 

a nunfcer of key terms and phrases utilized by such authorities, the follow- 

ing composite definition has been constructed: 

Evaluation is the systenatic process of: (1) ascertaining what in- ^ 
foimation is needed, (2) objectively collecting and analyzing through 
formal means (e.g., measurement, criterion, and statistics) the empir- 
ical and logical data related to the infoimation needed, and (3) re- 
porting the infoimatiori in a functional foiro which will enhance the 
judgmsnt of deo|s ion -makers in selecting among decision alternatives. 

The above pneral definition contains nine key phrases which require 

further definition. . These nine key phrases and their definitions are as 

follows: , ' / 

1. Systematic Process * Evaluation involves coordination of contin- 
uous and recurring or cyclical activity. It is an orderly and 
sequential set of numerous. methods and techniques. (Stufflebeam, 

et al, 1971, p. 40; Wysong, 1971, p. 48; ^fa^yland State Department 
of Education, 1972, p. 91; Tyler, 1969, p. 245) 

2. Ascertaining Needed Information . One of the first steps involved 
in tlje evaluatipn process is, the identification of required and 
useful infonpation. Delineation of such information must be made 
in consideration of the altfenative decisions that have to be made 
and the values or criteria which will be applied to them. Confir- 
mation and ^identification of appropriate information requires 
interaction with the client. Both the evaluator and the client 
are equally responsible for" the identification of needed informa- 
tion. While the decision-maker (client) determines the nature of 
the domain to be evaluated, it is the task of the evaluator to 
"point out inconsistencies, difficulties, or additional data that 
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might inodify the decisicn-Bjater's vi«*5 m iStm nievance of cer- 
tain outcomes'* (Alkin^ 1970^ p. 1)« Ibe ewlntifla in essence 
"critiques the client's fbmilations aad pninses alternative 
versions'* (Stufflebeam, et al, 4971, p., 42). It is iiyo r ta nt> 
hov/ever, that the evaluator be carefbl not to off-balance his 
res-pun sibility to the exclusion of the decision-saker*s ooinicn. 
This would result in a valuable loss of bbjectarity; (Stufflebean, 
et ai, 1971, pp. 41 and 42; Alkin>"1970, p. 1) 

3. Objectively Collecting and Aialyzin^ Iirfon^ These are 
processes which make information available through such fcroal 
meanSi^ as measurement, criteria^ and statistics. These are the 
technipal aspects of evaluation which nust take into account 
such criteria as internal and external validity, reliability, 
objecti\^ity, and efficiaicy (these tents will be defined in the 
latter part of this chaopter) . Throu^i careful analysis, col- 
lected data can be broken down ijnto\£oTO in ^diich the nature, 
significance, and relationship of attained variables can be de- 
termined. The tem "objectivity*' iiqp^lies 1±at the variables to 
be treated are handled in a uanner indq^eodent of personal re- 
flections or feelings. (Stufflebeam, et al, 1971, p. 42) 

4. Fprmal Means^ e.g., "Mgasurement, Critericgi, StatisticsJ >ieasure- 
^ ment is the assignment of mmexals to catitifes according to rjles, 

and such rules usually include the spedficatiai of sacqple ele- 
ments, measuring devices and conations for adoinistermg and 
scoring the measuring devices. A criterion is a rule by vfaich 
values are assigned to alternatives and, optimlly, such a rule 
• includes the specification of variables for Beasureawnt and stan- 
dards for use in judging that ^ch is seasared. Statistics is 
the science of analyzing and interpretiiig acts of neasureraents^'' 
(Stufflebeam,' 1973, p. 21) - 

5. Empirical and Logical ^Data, i.e. , IiiforBation> Infoimtion helps 

to rfiriiirfi^thfi nnrftrtflinty rW H^ri^s-inn^^kipf? baVC^vhoi selecting 

among^ alternatives. Infonaaticn consists of •descriptive and in- 
terpretive .data about entities and their relationships, in tenas 
of some purpose" (Stufflebeam, et al, 1971, p. 41). It can orig- 
. inate factually throu^ experiinental encounter or be induced intu- 
itively in harmony ^^th sound reasming. (Sufflebeam, et al, 1971, 
p. 41; isysong, 1971, pp. 48 and 50).- 

6. Reporting the Information . One of the final steps in evalmtion 
involves giving the collected infonmation to the decdsion-iaaker. 
This involves fitting the information together into a formt that 
.'T)est serves the purposes of the evaluation. "(Stuffleb^tm, et al, 

1971, p. 42) 

7* Functional Form . Ijhless the information is rqwrted in a foxtsat 
v/lth functional form, the decision-waker will jnot be able to taike 
optijTium us^ of the evaluation. In order to & j&joctional, eval-^ 
nation findings must be reported in a form iduch is not only 
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readable and understandable to the evaluator and decision-maker, 
but ^Iso related to" the values arid criteria i>^iich were, identi- 
fied and mutually agreed upon by both the decision-paker and 
the evaluator. In addition to the considerations ju^ mentioned, 
Stufflebeam (et al, 1971, pp. 42 and 43) further points out that,, 
in order to be .functional, information "must satisfy three classes 
of criteria: scientific (internal validity, external validity, 
reliability, and objectivity), practical (rejevance, importance, 
, scope, credibility, timeliness, ahd. pervasiveness), and'prudential 
(efficiency)." These classes of criteria will be discussed in 
•more detail in the latter part of this chapter. 

8. Enhance Judgment of Ijecision->fakers . 'Tlat^ cannot make the 
decisions for people) . , evaluation still requires people to 
make judgments and decisions" (uysong, 1972, p. 48). The ulti- 
mate purpose of evaluation is to provide a irational basis thklr' 
will increase the \^alue or worth ^ of the judgment of evaluator^ 
and decision-makers. Stufflebeam (1973, p. 21) defines judg- 
ment as "the assessment of values to alternatives." He also 
defines judging as "the act of choosing among. . .decision alter- 
natives, hence the act of decision-makings. "(Stufflebeam, et al» 
1971, p. 334) ^ * . 

There seems to be some^ontroversy concerning the evaluator 's 
role in judging. Some feel that participation on the evaluator's 
part in judging "destroys his objectivity and, hence, his utility," 
while others feel that the act of judging is a valid part of his 
role. (Stufflebeam, et al, 1971, p. 43) ^ , 

. In the case of evaluations involving contracts, the role of 
the evaluator in the act of judging <Jepends largely upon the terms 
of agreement between the evaluator and the decision-maker. 

,\fysong (1971, p. 48) has one more consideration concerning 
judgment which seems to merit mention at this point: "Evalua- 
tion is 'not a means of merely labeling a program as being ^good' * 
* or 'bad. ' Sjuch a label is usually not helpful in promoting pro- 
gram development and ijiprovement and debris ion-mal.ing. It gives , > 
no direction for future change." ^ ' 

9- Selecting Among Decision Alternatives . Once the necessary informa- 
tion is functionally provided, decision-makers are then better 
equipped to make preferences among two or more optional courses, 
of action. Among such courses of action, there is usually at 
least one choice requiring some foim of altered action. From 
the standpoint of education, altered action can especially be 
an important consequence of evaluation, considering the fact ^ 
that "educational improvanent occurs only as a result of some 
altered action ."(Stufflebeam, et al, 1971, p, 40) 
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Figure VI contains a chart designed by Nelson (1973, taken from speech) 
which illustrates four ijnportant concepts that decision-makers might ^-ant 
to consider in selecting among optional courses of action* The' concepts^, 
as depicted in Fi^re VI are as' follows: 

1. The selection of one course of action has a tendency to limit 
^the decision-maker from the many other plausible alternative | 
^'courses'^of action that are available. f: 

i 

2. Some choices limit future cop-ses of action more than others. 

3. No one choice presents an absolute answer to the problem or 

. , opportunity. And/or, %{. 

4. The probability of success of the alternatives are not fully 
known. 



V 



PROBLEM OR 
OPPORTUNITY 



Figure VI 
PiX>cess of Deci&ion 'Making 




KEY: 



= CHOICE 



\ ) .COURSE OF 
_±r = ACTION - 



=» END OF . 

COURSE- OF 
. ACTiaN 



The process of decision-making with all its optional courses of action 
can generally be broken down into four basic decision alternatives that 
decision-makers ultimately face when Concerned with the future orientation 
of their program. As previously suggested in Figure VI, these four ulti- 
mate decisions entail prerequisites of an infinite number of sub-decisions 
or considerations. 
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Figure VII illustrates the four basic alternatives vAich can be considered 
options to choose from by decision-makers. The four alternatives are as follows 



1. Initiate program efforts and begin generation of necessary feed- 
back information. ■ - , 

2. Terminate the program efforts made thus far "and, consequently, 
end future program progress as originally intended. 

3. Modi fir the program in some way, e.g. , by changing such things as 
precedes, inputs, or even the standards for the intended objec- . . 
tives or outputs. 

4. Accept program outcomes and/or present progress and continue up- 
coming processes as scheduled. 

-In an attenpt to briefly sum up ^iHkt has been presented thus far on the 

nature and definition of evaluation, the following two sinplified definitions 

are presented at this point: "Evaluation, in short, is the common sense of 

learning from experience. In the final step, we seek, through research, an- 

s;v'ers to such questions as 'How did we do; would we have been better off if 

we had tried something else'" (Cutiip, 1971, p. 129). In general, "evalua- 

tion may be thought of as the processes that are involved in measuring the 

coiisequences of goal-oriented action" (Griessman, 1973, p. 90). ^ < 

Figure VII 

^ B^ic Deeisdon Alterfiatives 
' Tlnitiate^ 0 ^ ' -i ' 



Feedback 
Information. 



Hie Integrated Evaluation Design 



Four Stages of Prograip Inproveinent for Career Education . 

After reviewing several sources (Alkin, 1970, pp. 2 and 3; Stufflebeam, 
1973, p. 22 and 1971, pp. 79-84; Caldwell, 1973, pp. 35 and 36; Randall, ^' 
1973, pp. 43-46; Coster and Morgan, 1969, pp. 6-13; Tyierl,'^1969, pp. 24?- 
^ 259) concerning educational change and program development, it was, dis- 
covered that most educational programs, including career education, evolve 
through definite cycles which can be categorized into distinct develop- 
mental stages. By ^integrating a variety of information on .program devel- 
opment and deci^on areas from the sources just mentioned, together with 
a review of in-fprmation on career education programs, the researcher has 
identified four major stages that many career education programs seem to 
follow through in the process of establishing program improvement. These 
four states are as follows: 9 fl) ijeeds assessment, (2) program development, 
(3) program, implementation, and (4) outcomes assessment. These four devdl- 
opmental stages, which actually represent a cycle of change or improvement 



within educational programs, can be placed upon a continuum as depicted 
iti Figure VIII. 



>^ Figure VIII 

. Career Education's Four ^fejpr Stages 
of Program Improvement 



Needs 
Assessment 




Program* 
Development 










Program 
Inplementation 



Outcomes 
Assessment 



Within each of the developmental stages, a sequence of events takes 
place. These sequential, events which are fcommon to each of the stages 



III-6 



are illustrated in Figure IX (p. III-8). As Figure IX shows, each stage 
of program improvement involves: (1) the capture of established inputs 
which *a|e derivi&d from the educational domain or environment in which the 
school system is part of, (2) a process which includes all transactions 
that are necessary to conplete the transition from .raw i^ipuf material to 
actual output, and (3) the actual output which is tenroorarily stored and 
then eventually circulat3d into the educational domain* for further use. 
Figure X (p. 111-9^ was constructed by placing the components of Figure 
.IX (p. III-8) into the continuum of Figure VIII (p. III-6). Descriptions 
of each of the stage: of program inprovement for career education, in- 
eluding appropriate reference to the various inputs, processes, and out- 
puts as depicted in Figure x'(p. lil-9) are as follo^vs. 

Needs Assessment . IXiring this stage of program improvement, motivation for 
program change is initiated. (Refer to the model in Figured, p.. 111-9) 
Inputs > The inputs for this stage of development would generally 
include such things as the involvement of consultants, community^mem- 
bers, parents, educational staff (including local, stafe,and fedetal 
levels), students, money, and the use of school and coiMunity facilities. 

Processes. The major processes occurring within this stage are: 
(1) the definition of the educational domain to be served, (2) the 

delineation and priority ranking of the needs of the educational do- 

'I 

main, and ,(3) the identification of the problem areas needing inprovement 

/ 



* The tern, educational domain^ as it is used in this paper, is defined a*^.; 
those resources (including people, school, and community facilities) -which 
have authority or control to govern or influence the decision-making pro- 
cesses upon which educational objectives, policjLes, criteria, and standards 
for a specified educational domain are established. 
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Figilfe IX 
Input - Process -^Output Chart 
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Figure X 

Coinbined Component Chart for the Four Stages of Program Inprovement 
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and the consequent establishjDent of specific program objectives that 
ave explicitly stated in terms of observable or measurable behavior. 

Outputs. The actual output would be an agreement among needs 
assessment participants pertaining to: (1) the definition of the edu 
cational domain to be served, (2) an established priority ranking of 
its needs, and (3) an identified list of objectives that somehow need 
to be implemented. 

\ 

The objectives of one program of career education will lasually differ 
in a v^ariety of \vays from the objectives of other programs due to the 
tendency of different educational domains to delineate ^d rank needs 
-differently. In order to give the reader some suggestions of possible 
educational needs and consequent objectives that could be established 
vdthin career education programs, a synopsis of selected career education 
goals for schools developed by Keller (1972, pp. 16-27) and a list of 
eight elements, along \dth eight desired outcomes identified by Ohio 
Stat(5's Center for Vocational and Technical Education (Miller, 1972 a, 
pp. .20-23), are listed in ^pendixes A and B, respectively. The synopsis 
by Keller was developed through detailed examination of a variety of ca- 
reer education handbooks, guides, and instructional units and the elements 
and outcomes by Ohio State were developed for use in their Comprehensive 
Career Education Model (CCBf) project. . 

Program Development , This stage, in essence, focuses on the development 
of a means to achieve the ends identified in the needs assessment. (Refer 
to 'the model in Figure X, p. Ill -9). 

Inputs . The major inputs used in this stage of improvement are: 
(1) the priority ranked list of objectives obtained from the needs 
assessment, (2) the time and effort of consultants, community 

IIMO 



iDembers, parents, students, and educational staff, including local, 
state, and federal levels, (3) money, and school and community 
facilities and materials. 

Processes . The process consists of designing a program which 
can best accon^)lish the objectives established in the first stage. 
In-process functions that are performed in the program development 
stage include: (1) staff and public orientation to the needs assess- 
ment. Including in-service .training, curriculum planning workshops, 
etc. , (2) development and specification of alternative strategies and 
procedures for future implementation, (^^^^^ev^lop^ of criteria to 
select the most feasible procedural alfemative, (4) specification^ 
attainment^ and preparation of needed resources, and (5) budget alloca- 
tion, scheduling and organizing of all available resources* 

Outputs . The general output resulting from the process of pro- 
gram development is a set of operating guidelines which include: 
(1) a selected procedure that specif ies , within the liinitations of 
available r^sow^^es, a plan of action for implementing the identified 

objectives and (2) an organized schedule of resources which compli- 

« 

' ments. the work to be perfomed. 

Program Implementation . This is the action stage of programmed ijiprove- 
iiient vAiere the procedures , strategies, and resources of the programming ^ 
stage are operationalized into activity. (Refer to the model in Figure 
X, p. III-9). 

•> * 

Inputs* The inputs for this stage include v:he procedural ap- 

ft 

proach selected in the program development stage with all its various 
instructional strategies and^ attained resources. 

Jr ' > V / 
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Processes . The general process involved during this stage is 
the actual- placing of the plan of action, as prescribed in the pro- 
gram development stage, into action. 

Outputs^. JXiring the implementation stage, the objectives that 
were earlier" established in the needs assessment are finally actual- 
ized to a certain measurable degree. Therefore, the outputs of im- 
plementation are dependent upon: (1) the expected student competencies 
and outcomes that were delineated in the first stage and (2) the 
initial competWcies of the students enrolled in the first stage. 
, JOutputs obtained in the foim of student gain or gro^rth would gener- 
ally fall under the classifications of the cognitive, affective, and 
psychomotor .domains . 
(Xitcomes Assessment . This is the final stage of program improvement where 
decision-makers^ ask the question, "How did we do, ivX)uld^ have been bettei 
off if we had trred something else?" (Refer to the model in Figiire X, 
p. III-9). 

5 

. i Inputs . The inputs o£ this stage consist of information concem-^ 
ing all the processes and outcomes that developed during each of the 
preceding stages. , \ , • 

Processes . The general process involves £in4ing out to what ex- c 
tent the desired outcomes established ^in the needs assessment stage 
have been achieved.. This is basically done by focusing on how the 
actual outcomes of the implementation stage relate to: (1) the in- 
tended outcomes that were determined ^in the needs assessment stage, 

(2) the effectiveness of the plan of action and the resources that 
were selected and attained in the program development stage, and 

(3) the efficiency in which the plan of action and resources were 
♦operationalized in the implementation, stage. 

m-12 
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Outputs . The outcomes o£ this stage consist primarily o£ infor- 
nation pertaining to the effectiveness and ef:giciency of all the 
phases of the program's fiill cycle. These are the products that 
can, m essence, perpetuate or re-cycle ^e process of program im- 
provement because thev offer the substantial information and criteria 
that decision-maters need in order to select among the three ultimate 
decision alternatives of: (1) affecting change in areas that need 
ijT^/rovement, through program modification, (2) accepting program out- 
comes and progress and continuing processes, as scheduled, o:^ (3) 
teminating program efforts. 

In order to summarize the four stages of program inprovement and bring 
their focus closer to the perspective of career education, the four stages 
of program improvement have been inter -related with a slightly modified 
edition of the ten action steps for iirplementing career education devel- 
oped by Hoyt (et al, 1972, p. 150) as shouTi in Table II (p* III-14). ' By 
the time action step seven or the iir^jlementation stage for career educa- 
tion is reached, the ten substeps required for implementation, also devel- 
oped by Hoyt (et al, 1972, p. 170), as shown in Table III (p. Ill -15), should 
have been completed. 

Four Stages of Evaluation for Career Education 

Earlier in the previous section concerning the stages of program im- 
provement for career education, it was mentioned that a full cycle of 
change or improvement occurs within career education programs and also, 
that this cycle could be placed upon a continuum of career education's 
inprovement cycle (Refer to p. III-6). It is recommended, however, that 
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Table II 4 

Stages of Program Improvement for Career Education 
Inter-related v:ith the Ten Action Steps for Implementation 

Stage I " Needs Assessment - 

1. Organize the appropriate interactive net\vork of interested indi- 
viduals and groups (e.g., school staff, parents, community mon- 
bers, and evaluation personnel) . 

2. Put the evaluative process in operation to determine ho^^ well the 
system is working. (TKis process should contin^ue throughout all 
four stages.) 

3. Gain an understanding of the concepts of career education and " 
establish appropriate educational objectives. 

* 4. Study the current educational system to determine the changes, 
necessary to turn It into a true career education system • 

5. Create a feedback system to use evaluation findings to adapt and 
improve the career education program. (This process should con- 
tinue throughout all four stages.) 

Stage II - Program Development 

6. Inventory and marshal all available resources. 

7. Begin planning the career education system most appropriate for 
your comrrajnity. 

8. Seek the cooperation of all necessary organizations, institutions, 
and individuals. 

Stage III - Program Implementation 

9. Implement the system (note Table Hi, p. III-15). 

Stage W - Outcomes Assessment "\ 

10. Hake provision for a program of maintenance to sustain the vital 
p^rts of the system and the initiative, and tie these activities 
into the interactive network (i.e., (1) determine the effective- 
ness and efficiency of all phases of the program's full cycle, 
(2) maintain and recycle the vital aspects, and (3) modify or 
exclude the inefficient and ineffective aspects) . 
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• ^ Table III 
Ten Substeps Required for Iicplonentation 

1. Contact ;dll have been made xdth state edtSCational agencies and 
available guidelines and state polic)' endorsed by local education . 
authorities. ^ " 

2. A compendiun o£ educational objectives \dll have been established 
at every Igvel where career education is to be .iinplemented in the 
first target year. 

3. The process of career education \\ill have been examined l?y school 
personnel and community resource persons and experimental programs 
tried and evaluated on a limited basis. 

4. An evaluation of existing school programs and activities has been 
initiated, resulting in recommendations for changing specific parts 
of the curriculum. 

5. Professional personnel and ccmmimity educational resource persons will 
have formulated new strategjLes and programs and teachers will, have re- 
ceived whatever in-service retraining is required. 

6. Ke\7 programs and activities ^dll be ready for implementation. Detailed 
descriptions xdLll be available for every new course or learning activ- 
ity' concerned with career education. ' 

7. Estimates of faculty time and the need, for special equipment and facil- 
ities will have been deteimined and dEunds vnll have been provided- 

8. Detailed budgetary procedures will have been followed by determining 
the cost of new programs and activities. 

9. Cost-saving measures will have been recorded, particularly from those 
courses, programs, and activities that may have been deleted from the 
original program, of instruction. 

10. Follc\dng an analysis of cost and benefits relative to new programs 
and activities, alternative strategies will be examined to permit 
the phasing of courses, programs, and activities of high priority 
in^o immediate operation^ while assigning new implementation dates 
to activities of lower priority. 
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evaluation begin as early as possible in program development and continue 
throughout all consequent phases in order to keep the career education 
program constantly tested for continuous inprovement (^iaryland State 
Department of Education, 1972, p. 91; Tyler, 1969, p. 24L; 1^/song, 1971, 
pp. 48 'and 52). • ' 

♦ 

Stuff lebeam (et al, 1971, pp. 215-235) has identified four general 
kinds of evaluation that Kere designed to yield information pertaining to 
four classes of decisions that were established as part of the Sti^flebeam- 
Cuba CIPP model. These four types of evaluations could be readily adapted 
to the continuum shown in Figure VIII (p.. III-6) by placing them along* the 
cycle of program improvement for career education as illustrated in Figure 
XI (p, III-17). These four classifications of evaluatipn, called context, 
input, process, and product evaluation will be used in this report to des- 
cribe the four stages of evaluation that Could occur during each of the 

stages of program inprovement for career education. 

o 

Notice in Figure XI (p. III-17) that: *(1) the decision to initiate 
program efforts precedes the needs assessment stage, (2) the general pur- \ 

pose comnon to each of the four stages of evaluation, is to supply decision- 

/ 

makers with feedback information that will aid them in selecting among the 
three ultimate alternatives of either terminating, modifying or continuing 
the various stages of program improvement, and (3) the last stage of pro- 
gram improvement is the only stage that allows the. option of deciding 
whether or not to re- cycle (R.) the entire program of improvement. 

Figure XII (p. III-18) shows the typical sequence* of events that takes 
place within each of the four stages of evaluation. Like the stage of pro- ^ 
gram improvement diagrammed in Figure IX (p. Ill- 8),^ the activities within 
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each of the four kinds of evaluations involves a relationship between in- 
puts, processes, and outputs. In the case of each of the stages of evalu- 
ation, however, this relationship involves only the gathering, analyzing, 
and reporting of information pertaining to the inputs, processes, and out- 
puts, rather than the actual delineation, attaining, and operationalizing 
of these events. 

Generally speaking, each of the four stages of evaluation consists 
of the following sequential activities : (1) information pertaining to 
inputs, processes and outputs is collected, (2) comparisons bet^veen actual 
outputs and th^ intended output standards (as detemlned by the educational 
domain) are made, and (3) the results of number one ani t^io above, are syn- 
thesized, analyzed, and then reported functionally in the form of feedback 
and discrepancy infori];iation. Descriptions of each of the stages of eval- 
jLiation for career education are as follows. 

Context Evaluation . This ik^the stage of evaluation where criteria is 
established, for the development^ objectives. Its general purposes are: 
• 1., To define the boundaries of^^the, educational domain to.be served. 

2. To describe the actual conditions of the educational domain. 

3. To detemine the proposed ideal conditions fbr the educational 
domain. 

4. To identify the unmet needs, of the educational domain by deter- 
mining discrepancies bet^veen actual and ideal conditions. 

5. To ascertain barriers causing the needs from being met. 

6. To define the values, attitudes , and priorities of the educa- 
tional domain • 

7. To specify objectives which indicate how much change is actually 
wanted. — 
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Categories and examples of discrepancy information which may indicate 
that career education needs are not being met are: 

1. Academic hbtivation - lack of student interest in school subjects. 

2. Instructional Relevancy - lack of student knowledge of the re- 
lationship of instruction to the real life implications' of the 
world of v^vrk. ' ^ , . 

[I * 

3. Career Avargness - lack of student exposure to the wide range of 
• educational opportunities, career options^ and consequent life 

styles that are available in the wtorld of \vDrk. 

» 

4^ Orientation to the World of Work - student unfamiliar ity .with 
the nature of our work-oriented society with all of its work 
attitudes, promise, and problems. 

5. Self Asvareness - lack of student self-understandings (e.g. of 
their own interests, aptitudes, and attitudes) and knowledge of 
how the self relates to the different styles of life which accom- 
pany different types of work. / ? ^ 

a. Interests - lack of knowledge abcjiut one's o;^/n personal pref- 
erences and concerns, e.g., motivational influences that ' 
affect skill performance. ^ ^ ^ 



b. Aptitudes '- inability to learn the skills involved in various 
kinds of work. 

c. Attitudes - personality characteristics and value orientations 
which tend to indicate an inability to satisfactorily relate 
to self, others, situations and ideas, as evidenced by a lahk 
of such personality traits as ^elf-identity, se^f-esteem, / 

' prestige, and confidence. ' ' ' - , 

" f ^ / 

6- Career Planning Skills - lack of student knoiVledge necessary for 
enployability development, i.e., an inability to perform the [ 
skills essential in preparing thejself to successfully relate 
to and function within the world pf work. 

/• Decision-Nfaking Skills - inability)' to satisfactorily select 
(tentative or otherwise) a career among several, occupational 
alternatives, through the careful and systematic consideration 
and integration of such relevant] factors as self-awareness and 
career awareness. ^ 

8. Job Readiness - lack of job entry-level skills among students 
who exit school; a possible consequence of insufficient "hands- 
on" work experience. 
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9. Demographic - evidence of unusually high rates of such statisti- 
cal records as- school drop-outs, absenteeism, unemployment, criine, 
drug addiction, and recidivism. , * 

10. Barriers - presence of. factors that restrict or -impede, the effec- 
tive implementation of pareer education, e.g., the presence of, 
unequal career and educational opportunities among students of 
(differing socio-economic, racial, and cultural backgrounds. 

' Input Evaluation . The general purposes of input evaluation are: 

1., To provide information pertaining to all the ayailable resources 
and alternative strategies that could be used within the educa- 
tional domain in order to achieve the objectives stated in the 
needs assessment stage. ^ - ^ 

2. To provide criteria that will assist decision-makers in selec- 
ting the best plan "of action. 

Discrepancy information is obtained during input evaluation by com- 
paring the alternative procedural designs or plans of action with the 
criteria of ^^ifiUb^^te^ requirements' demanded by the objectives estab- 
lished in needs assessment. 

Process Evaluation . This stage o^ evaluation occurs once the selected plan 
of action* and tlie attained resources h^ve been put into operation, the 
major purpose of process evaluation ds to provide information^ \diich will 
assist appropriate educ^tioi^al staff in the continuous process of monitoring, 
record- keeping, refining, and controlling of implementation plans and pro- 
cedures, i.0., to detect and' predict present and potential defects in re- 
sources, procedural designs, and metJiods of implementation. The record- 
keeping task in process evaluation is important, not only for describing 
what is actually taking place during implementation, but also for later 
use in interpreting actual process outcomes. 

Discrepancy information is obtained during proctss evaluation, by 
comparing differences betiveen ^he actual procedural performance and the 
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intended procedural design (i.e., by comparing actual events with intended 
process activities). 

Product Evaluation ^ Thj.s stage of evaluation occurs after all the stages 
of program inprovement have completed a full cycle. The general purpose 
of the product evaluation is to provide information which will help deter- 
mine to what extent the desired objectives established in the needs assess- 
ment stage have been achieved. According to Stuff lebeam (et al, 1971, p. 
232) this generally involves the follb;\ang activities: 

1. Devising operational definitions' of. objectives • 

2. >Jeasuring criteria associated with the objectives of the activity. 

3. Comparing these measurements with pre;determined absolute or 
relative standards. - ^ 

4. Majjing rational interpretations of th^ outcanes, using the 
recorded context, input," and process information. 

The end product of this stage of. evalua1;ion results in an interpreta- 

p. 

tion of the overall effectiveness of the entire cycle of program improve- 
ment. , - , ^ 

* *» 

The discrepancy information obtained in this stage of evaluation is 
obtained by comparing information pertaining to actual program outcomes 

against the intended outcomes determined in needs assessment. 

* ♦ 

The Total Evaluation System 

Figure XIII (p. III-23J illustrates how a typical section of the total 
evaluation system looks when the evaltiation proces? chart,' the input- 
process-output chart, and the basic decision alternatives are all combined.' 
The sequence of events or activities that occur within this typical sec- 
tion of the total evaluation system are as follows: 



■ ■ it 
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' Figure XIII 

Evaluation Process Chart, Input -Process -Output Chart 
and the Basic Decision Alternatives Combined 
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!• iTifonnation and resources procured from the educational domain 
are stored as actual inputs ^dthin the appropriate stage of pro- 
gram iiiprovement. 

- 2/ Information pertaining to the actual inputs is collected and 
stored by the appi:x)priate stage of program evaluation. 

3. The actual inputs are processed in a fashion appropriate to the 
stage of .prqgiiam iirrorovement* 

4. Information pertaining to the actual process is collected and 
stored by the appropriate stage of program evaluation.' 

5. The actual outpr^s are stored until a decision can be made con- 
cerning their future destiny.- 

6. Information pertaining- to actual outputs is collected and stored 
by the appropriate stage of evaluation. 

ft 

7. Con5)arisons and discrepancies bet^ceen actual output information 
and the criteria of the intended output ^tsndards are made by 
t^e appropriate stage of evaluation/ 

8* Information pertainijig to actual inputs, processes, and outputs 
are synthesized and analyzed, along wi:^ the comparisons be- 
tween actual output information and the"criteria of intended 
output standards by the appropriate stape of evaluation. 

9. Thp combined infomation resulting from the appropriate stage 
of evaluation is reported functionally to decision-makers in 
^< the form of feedback and discrepancy infoimation. 

10. fin ultimate decision is made to do one of the following: 

a. Terminate program efforts. ^ 

b. . Inprove the actual outputs by re-cycling them and modifying 

the inputs and processes that created them. 

c. >bdify the criteria pertairring to the intended output stan- 
dards in order that they conform more realistically with 

' the actual outputs. . ^ 

d* Accept program outcomes and present progress and continue 
up-coming processes as scheduled. c 

Figute XIV (p. .111-25) represents a model i;vhich shows what the total 
evaluation system looks like when the typical ^valuation section shown in 
Figure XIII (p. III-23) is placed along the continuum depicting the cycle 
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F i qure. XIV 
THE TOTAL EVALUATION SYSTEM 
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of program improvement that was illustrated in Figure XI (p. HI- 17). There 
are four features of the total evaluation system \ihich can now be seen in 
the model shoioi in Figyre XIV (p. III-2S). "These four features are as 
fol]^ows ; 

1. The final stage qf the fullncycle of program ijnprovanent is the 
only stage that eventually faces the decision of whetlaer or, not ' 
to continue the career education project by re-cycling the entire 
program of, improvement • 

2. The flow of inputs, outputs, and information is not only derived 
from the educational ciomain during each stage of program improve- 
ment, but is also eventually returned to the .educational domain 
for further use after each stage of program improvement and 
corresponding stage of evaluation is completed, 

3. Each stage of program improvement has the decision capabilities 
of either: 

a. modifying the context criteria of intended output standards, 
or 

b. modifying actual outputs by re-q^cling them to any of the 
stages of program improvement that precedes the re-cycling 
decision. 

4. . The process of evaluation begins at the earliest possible time 

of program development and persists throughout all consequent . 
phases to offer continuous opportunities for constant program 
inprovement of career education. 



Tasks and Considerations Involved in 
Developing and Administering the Evaluation Desij^in 

Hiis section of Chapter III is devoted to suggesting various tasks and 

considerations that will be necessary in further developing the evaluation 

system just presented. In order for the ev.aluation system to be operationally 

functional, there has to be defined specific tasks aiid considerations that 

are needed for its implementation. Different programs require a different 

series of tasks and considerations for their evaluation for the same reason 

that different programs have different needs and objectives. It is not the 
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intent of this section to suggest specific tasks and considerations that 
would be unicpjely suitable to all career education programs. Such a ven- 
ture is beyond the mission of this paper. However, in an effort to make 
the previous evaluation design operational without over-generalizing, the 
following sub-sections are offered: (1) Suggested Variables of Information 
iNTiich ^fay be N'eeded for the Four Stages of Evaluation, (2) Suggested Msthods 
and Instruments Vhich May be Useful in Collecting Information, (3) Suggested 
^fethods V.hich ^fay be Useful in Analyzing and Treating InfoniQtion, (4) Sug- 
gested Procedures and Formats Uliich ^^ay be Used to Functionally Feedback 
the Results of Evaluation, ard (5) Suggested Criteria I'Mch ^fey be Useful 
in Assessing the Adequacy of Evaluation. 

As was alluded to earlier, the tasks and considerations listed within 
these sub-sections are only offered as suggestions or alternatives and 
should not be considered either exclusively or inclusively applicable, for 
the implementation of- all prograJi evaluations. For additional informajtion 
pertaining to suggested tasks and considerations in conducting evaluations, 
it is recommended that the reader refer to Appendices E and F. Appendix 
E contains a detailed, sequential taxonomy of evaluator tasks and proposes 
an equally detailed' sequential list of specialist roles in evaluation. 

/ 

These evaluator tasks and specialist roles were developed by Ow^.os (1973, 
pp. 73-79). In presenting the taxonomies, 'Oivens was careful to point out 
that his roles and tasks were "proposed as illustrative types" and that 
the appropriate use of the roles and tasks will vary from one setting to 
another, depending on such factors as size, budget limitations., caliber 
of existing specialist, types of evaluations to be performed, and the organ- 
ized structure of the school district. / 
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Appendix F contains a "Table of Contents Based on a Work Breakdoim 
of Evaluation Tasks and Activities." This table of contents, developed 
hy Stufflebeam (et al, 1911, pp. 203-205), also sequentially lists tasks 
and considerations which may be used in conducting program evaluations. 
.Detailed descriptions of these tasks and activities may be found by re- 
ferring to pp. 139-213 in Stufflebeam's book. Educational Evaluation and 
Decision'•^faking (refer to list of references) • 

Suggested Variables of Information \^^uch May be Needed for the Four Stages 
ot Evaluation " 

To provide information necessary for decision-making, "'the evaluator 

needs to wk within a framework th^t offers a vide range of potentially 

relevant variables'* (Stufflebeam, e\ al, 1971, p. 241). To meet such 

••specifications," Stufflebeam (et al, 1971, p. 24^) recommends the EPIC 

Evaluation ^bd^l (developed at the EPIC Evaluation Center in llicson, 

Arizona) as a •'most useful heuristic device to reveal combinations of 
y 

variables leading to a more complete description and analysis of the 
instructional program. •• As illustrated in Figure XV (p. TII-29), the 
EPIC Evaluation ^bdel consists of three major sets of variables, which 
are further divided along three dimensions into 14 basic sub-sets. "In 
/ considering the dimensions, analysis of variables is liinited only by the 

nature and scope of the program and the desire for sinple or complex 
analysis" (Stufflebeam, et al, 1971, p. 242). By measuring the results 
of interaction among the variables, "A description of the forces affecting 
program results is produced. . ." (Stufflebeam, et al, p. 242). For fur- 
ther assistance in the classification of the behavioral dimension, the 
taxomonies developed by Bloom (1958), Krathwohl (et al, 1964), and Simpson 
(1967) are recommended, 
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Figure XV 
The EPIC Evaluation Model 



INSTRUCTION 



Student 
Teacher 

Administrator 
^ Educational Specialist 

Famijy 
Communi ty 



o 
cz 



o 
2: 




\ \ V V 



III-29 

91 



As Stuff lebeam (et" al, 1971, p. 244) will attest, the evaluation model 

designed at the EPIC Center can sen^'e as a useful "starting point" for 

variable classification, however, "it must still be adapted to the needs 

of individual institutions." In an attempt to closer adapt the evaluation 

system that was earlier presented in the paper to the needs of career edu- 

cation programs, the following lists of major variables and general decision 

areas were identified. These lists will be separated and presented accord- 

ing to the appropriate state of program improvement for which they are 

\ 

designed to serve. * 

I. Needs Assessment Vari.^bles , With the help of Stufflebeam (et al, 
1971, pp. 81, 218-222), the following list of questions were ident- 
ified. These questions serve as examples of information which may 
be needed during tne process of context evaluation. 

A. Vlhat are the boundaries of the educational domain? 

B, IVho are the decision-makers to be served? 

C* What are the actual present conditions and characteristics of 
the educational domain? 

D. What are the conditions that are considered ideal for effective 
implementation of career education (as determined by the decision- 
makers of the educational domain)? 

E. What are the unmet career education needs of the educational do- 
main (as determined by identifyirig discrepancies between actual 
and ideal conditions) ?^ 

F. What are the barriers causing the needs from being met? ^ 

G. IVhat are the values, attitudes, and priorities of the decision- 
makers of the educational domain? 

H. Do the decision-makers want the unmet career "education needs 
to be met? 

I. If so, then how many of the unmet career education needs are 
the decision-makers ^dlling to meet and to what extent do they 
want to meet them? 



* Refer to pages III-20 ^d IIT-21 for a more detailed list of variables 
exemplifying and categorizing types of discrepancy information which 
may indicate that career education needs are not being met- 
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1. IVhat are the specific objectives (including their priorities) 
of the career education program going to be for the future? 

2. IVhat specific behaviors do the decision-makers want students to 
exhibit after their exposure to the career education program? 

Program Development Variables . Stuff lebeam (et al, 1971, p- 233) sug- 
gests areas of information that are needed by providing the following 
examples of general questions that could be asked during input evalua- 
tion: 

A. How familiar are all staff and community members^ who are to 
be involved in the program, with the behavioral objectives and 
general program as delineated in the needs assessment? 

B. Are the given objectives stated operationally and is their 
accomplishment feasible? 



C. Miat strategies already exist with potential relevance tor meet- 
ing the established objectives? 

D. How can alternative strategies be generated? 

E. IVhat are the potential goals and benefits of each of several 
competing strategies? 

F. IVhat are the operating characteristics and effects of competing 
strategies under pilot conditions? 

G. Is it logical to believe that a given strategy can accomplish 
the specified objectives? 

H. Has this kind of plan worked in the past? 

I. Is the particular strategy based upon valid theoretical principles? 
J» IVhat are its underlying assumptions and can they be met? 

K. Is it legal? " \' 

L. V/ill it be rejected on moral grounds? . , ^ 

M. How much training will the staff require before they can effec- 
tively implement a giv^n plan? (Once trained, how effective 
were the various fonns' of in-service training?) 

N. Can a given strategy be translated into an efficient procedural 
plan? x> 



0. ^^hat specific procedures will be needed to implement a given 
strategy? 




P. 



How can existing staff and facilities best be utilized in the 
implementation of a new strategy? 
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Q. IVhat specific schedule o£ events, resources, and activities can 
best guide the strategy's implementation? 

R. IVhat side^ effects might a particular strategy produce? 

S. IVhat are the attitudes of students, parents, teachers, etc., 
toxvrard a particular strategy and what do they know about it? 

T. How should a particular strategy be administered, evaluated,-^ 
and reviewed at various levels? 

U. Itfh^t process .and product evaluation designs are required for 
efficient and effective strategy implementation? 

V. Following a pilot test, how feasible is Strategy institutional- 
ization? - *\ 

According to Stufflebeam (1973, p. 22), alternative procedural deci 

sions for input evaluation can* be assessed with the following variables 

^ in mind : 

1. Their resource (capability^-and availability in meeting the . 
situation) , time, and budget ( requirements . 

2. Their potential procedural barriers . 

3. The consequences of not overcoming these barriers- 
' - .4. The, possibilities andJ^osts of overcoming them* / 

5. Relevance of the designs to program objectives. 

^ 6. Overall potential d£ the design to.meet program goals. 

' . « * * 

III. Program Implementation Variables . The following questions,, derived 
from Stuff lebeam (et al, 1971, pp. 83 and 230) , serrve as general \ 
, suggestionsvof information needed' in ptocess evaluation: 

A. jAre the interpersonal relationships among staffs members and be- 
Itween staff an4 students conducive with the program ef:^orts? 

B. Are comnxmication chalhnels among staff members and between 
staff and students conducive 'With the program efforts? 

C. Were there any misunderstandings of kn agreement with the in- 
tent of the program by persons involved in and affected by it? 

D. Were the resources, physical facilities^ staff, and time schedule 
adequate as i^as calculated? (i.e.. How i^ell did they work, in 
|)ringing about the desired oiiitcomes?) 
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' E. To what extent has the intended plan of actio^, been carried out? 

.F. Should the staff be re- trained? 

G. Should new procedures be instituted? 

H. Should additional resources be sought?' 

I. Should responsibi]^ities be reassigned to staff? 

J. Shbuld the schedule be modified? 

IV. Outcomes Assessment Variables . The following list of questions, ob- 
tained directly from Stuff lebeam^ (et al, 1971, p. 84) smuggest the 
general decision areas or variable types of information that may be 
considered during product' evaluation: " - 

' Al Are the student needs being met? 

B. ^re we solving the problems as "intended? (i.e., Tb \^ai exteiit 
have the desirgd putcomes been achieved?,)* 

C. Is the project failing? 

D. Was the product worth the investment"^ , 

E* Has there been a significant gain in pupil achieveme^lt? 

Have we benefited by using the opportunity that ras presented 
to us? 

G. Has sufficient progress been achieved to warrant continuation 
of the project? 

H. Is the new project succeeding?' 

I. Were the results from projegt A better than those from project, fe? 

J. How effective and efficient were the processes occurring through- 
out all previous phases? - ' , ' 

K. Are there any positive 4ind/or negative by-products? e.g.: 
1. Has the project resulted in in^roved teaching competence? 



* Again refer to the list of variables that may be used to describe un- 
met career education needs located on p. 1X1-20 aild Iil-21. The informa- 
tion concerning these variables can be used as both pr<:- test measures 
(gathered during context,^ evaluation) and post- test meaWes (gathered 
during product evaluation) |> 
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2. Have school-gommunity relations been improved? 

3. Have students improved their self-concepts? > 
Information, pertaining to, Specific criterion measures for outcomes 

assessment may be found by referring to Appendix J, ^^itiple Criterion - 
Nfeasures for Evaluation of School Programs, by I4stfessel and Michael 

(1967, pp. 27, 931-943) • ^ , ' ' • 

* * ♦ 

Suggested ^fethods and Instruments IVhich May be Used in Collecting Informatiui . 
This portion of Chapter III mil be divided into tuo major parts: 

(1) >fethods of Collecting Information and (2) Instruments for Collecting 
Information. . * ' ' 

\ ^ V 

I 

Method^^ of Collecting Information . Thi^ major, part will be sub-divi,ded 
and presented in the following order: (1) Suggested Research Designs, 

(2) Suggested Types of Measurement Devices, and (3) Suggested Data Collec- 
.tion Methods and Techniques (this third sub-divi^idn will be presented and 
separatedvaccor.ding to each pf the four stages of evaluation^ that they are 
appropriate f or).^ / - ' . ' 

• Suggested Research Designs . According to Gilrain (1973,^ pp. 18 and 
19), the three research designs most often used in recent evaluation stud- 
ies are: (1) Experimental vs. GDntr9l group post-test only, (2) Experi- l- ^ 
mental vs. Control group, pre- and post-test, and (3) Solomon Four Group 



Design (note Append:^x G). Out of these three, Gilrain (1973, p. 19) states 

that ' . ,^ ' , 

''The strongest designs for the me'asurement of vocational maturity 
variables as they are affected by career education projects are: , 
(1) the ^Experimental vs. Control pre- and post-test employed by 
Bovee (1967) Corey (1969)f Goodson (1969), Guerriero (1967), 



III-34 ^ 



ERiC . 96 • 



' Jalkanerr • (1970) , Schnieding and Jensen (1968) , Shirts (1966) , 
Stevenson (1971), William^ (1967) and (2) the Solomon Four Group 
Design enployed by Asbury (1968) and Jackson (1971)." 

'In making some di,s^ctions between these two research designs, Gilrain 
further asserted that even though the ExpWijncntar VS. Control pre- and 
post- test is not as conplex as the SoloFion Four Group Design, it should 
still suffice most research design requirements. One distinct advantage, 
ho'A'ever, that the Solomon Four Group Design has over the Experimental 
vs. Control pre- ahd post- test is that it has the ability to distinguish 
the effect of pre- testing. 

The Experiir.2ntal vs. Control pre- and ipost- test involves the u:^e of an 
experimental group (the group which is subjected to special project treat- 
ment) and a control i,^oup (the group '.Aich is not subjected to the same 
special project treatment). Before the Experimental group is exposed to 
treatment, both groups are administered identical pre-tests^ preferably 
under identical situations. Ti.en, following the experimental treatment, 
each group is administered the saji;e instruoient as a po^t-te?t at lAich 
time gains^ analysis is perfomed to docermine whether or not any signif- 
icant change differences in subject gro\v.-h between the two groups .resulted. 
• According to Gilrain (1973, p. 19), the subjects for such groups ar? 

not usually random samples, but are rather '^naturally assembled collectives 

♦ 

such as students of the same elementary class or students of different 
grade levels, such as grade seven through twelve, whD^are exposed to the 
same teacher of a specific subject matter. 

The use of a cc^trol group seems to b^ a necessary element in the 
process of evaluation. Its pUrjDose as a discriminating factor in distin- 
guishi4g gain5 due to natural, maturation, as opposed to gains attributable 
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to the experimental effect of careW education' increases the Wudit^ 

of evaluation results. Gilrain (1973, p. 24) testifies fo this concept 

in the foilov.lng statement: 

' J'The use of a control group seems iin^ierative. It is my experience 
that both the control and experinenjal groups^make significant 
gains as a-nonrtal maturation procesk"*and the only way to measure 
the effect of the Career Education proj.ect is to compare experi- 
mental gains to control gains/' » 

In support of the research that Gilrain has provided thus far, Writes 

(1969, p. 85) offers the following features which he feels are^ essential- 

for 'The ideal experimental design for research on the facilitation 'of ^ 

vocational attitude maturity*': . ^ , ^ / 

^ ■ / 

1. Unrestricted range on the dependent vpri^ble. 

2: S's sampled from -the same population or stratified on rele- ^' 
vant control variables. - / 

Z. Sol^n Four Group Design^tp ^ssess .effects of pre-testing^' » 

4. "Gains" analysis of differences betu'een pre- and post-test 
scores. 

5. A relevant, potent, operationally defined independent variable , 
(career education) to which only the experimental gro%) is 
exposed. / * ^ ^ 

For more information that: ^(^) ,further describes the research designs 

ju$t presented and (2) describes additional research^ designs that my be 

# « ' ' 

useful in collecting information-, refer, to 'Appendix G. Appendix G wa§ 
^adopted from Dalen and Meyer (1966) by Isaac and^ Michael (1971) and con- 
tains the "^following eleven examples of research^degigns : (1) A "Poor"^ 
Research Design (fCo Control): The "One- Shot (^se .Study," (2) Minimal 
Control: Chie-Group Pre-Test^Post-Test Design, (3) Randomized Control - 
Group Pre-Test-Post-Test Design,«(;4) Randomized Solomon Four-Grou^ Design, 
(5) Randomized, Control-Group' Pre-TeSt-PostTlest Design, (6) Non-Randcsnized 
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Cojitrol-Group Pre-Test-Post-Test Design, (7) Counterbalanced Design, 
• * > 

(8) Qne-Group Time-Series Desigi^, (9) Control -Group Tiine-Series Design, 
(10) The 2 X 2 Factorial Design, and (11) Extended Factorial Designs. 

Suggested 'T>npe.s of Measurement Devices and Criterion Measures . Accord - 
ing to Gilrain (1973, p. 4 and V/estbrook, 1971), the three most commonly 
used devices and instruments to measure variables of vocational matuiuty 

are: - / . • 

• * ■ « 

^'(1) the partially structured interview used in Super's Career 
Pattern Study (Super et al, 1960); (2) the completely structured 
interview used in the Careejr Development Study (Gribbons and 
' Lohnes, 1968) to mea*3ure Readiness for Vocational Planning '(RVP); 
(3) the fixed ai-teniati\''e procedureused py Crites (1966) in his , 
50 trup-ralse questionnaire, entitled Vocational Development In- 
ventory (VDI)/* " - ' ^ 

, As. implied by tRe name, the partially structured i'r.;,ei/iew consists of 
non-predetemined question's and ans^vers. One disadvantage of this type of 
instrument is that the analysis of data is quite complex due to the fact 
that the interview format' is. non-standardized or non-qomparable. The struc- 
tured interview, on the other hand, does have pre -determined qjiestions and 
a pre -determined order of presentation. ^ Therefore, the ansivers of all sub- 
jects can be ^compared to each other and are somewhat easier to analyze. 
The nature of the fixed alternative choice limits the subjects to the 
choice of stated alternatives and, consequently, eliminates the possible 
error of variance caused by score differences in e^^aluating responses. 
In comparing the fixed alternative choice to either o£ the two types of ^ 
interview instruments, it would be safe to say that the fixed alternative 
choice instrCmient has the advantage of being 'less difficult and less ex- 
pensive to analyze than the free responses of the interview instruments 
O'/estbrook, 1971, p. 10) • ' • ' 
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. It -should be mentioned, however, that both the partially structured 

and the structured interviews have the distinct advantage of gathering 

free open-ehded informa.tion and, therefore, offer opportunities fox the 

examinee to eiipress a wider and, perhaps, more inclusive r^nge of informa- 

tion which is not always available from forced-choice instruments. 

In summing up the three approaches, Westbrook and Cunningham (1970, 

pp. 18 and 171-175) have tfee following comments: 

"Interview approaches to measuring vpdational maturity have, dis- 
tinct disadvantages which limit their usefulness and applicability. 
Collecting the data is time-consuijiing and scoring requires a grea't « 
deal of time from highly qualified personnel. Tests and inven- 
tories appear to be more appropriate for measuring' the cognitive » 
and affective aspects of vocational maturity because they are^more 
objective, more economical, .can be administered to Xarge samples 
and yield moret precise mea^rements." • ^ 

Based on a detailed review of literature (Crit^s, i966; Bathory, 1967; 

Dutt, 1968; Drahozal,- 1905^; Cfites, 1969; Harlan, 1964; Ilolloway, 1967; 

f . ' / ' ^ . ' " ^ - ^ 

Asbury, 1968f Cooter, ,196^;" Maynard, 1970; NfcCrystal, 1967; Miller, 1968; 

/ ^ f ^ 

/ ' 

Meyers, 1966;Wilst^Ch, 1967; Bovee, 1967; Gilliland,^ 19p6; Goodson, 1969; 

' . / ' ^ . " 

Guerriero, 1967: Jackson, 1971; Schnieding and Jensen, 1968; Shirts, 1966; 

Stevenson, 1971; Williams, 1967; Westbrook and Cunningham, 1970; Norton, « 
1970; Cor^y, 1969; Vriend, 1969; Gerstein and Hqover, 1969; Isaac, 1969; 
Jeppes^n, 1971; >fcCorfnick,. 1969; McSherry aiffl O'Hara, ,;L966; Schlesser., 
197^;' Rollings, 1967; Sturges, 1969; duper, et al, 1971; Westbrook, 1971; 
l^iison, 1969; Zikmund, 1971; Bradly, 1970; Jackson, 19^9; Hansen, 1967; 
Jalkner, 1970 and others) containing research and analysis of instruments 
used to measure the appropriate vocational maturity variables earlier men- 
tioned, Gilrain (1973, p. 12) came to the following conclusion: 

. .the instrument most frequently used to measure vocational 
maturity attitude is Crites' VDll/v^he instruments most frequently 
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used to measure the acquisition' of seH-knowledge, occujJational 
• . infbrmatiort, vocational' planning, and decision -making are multiple 
' choice tests, Likert scale inventories, job list recall, and essay 
quest iojis." , ' . - 

• . . . ' A . 

' ' htost o£ the types of instruments summarized by Gilrain can be machine- 

scorra, except for the job list and^^essay question. The job list seems 'to 

lend it5,elf to Hand-scoring quite readily and is supposed to discriminate 

'vrell bet^veen experijnental and control groups, as well /as bet^veen pre- and 

posf-testing results/ One. disadvantage Is ithat the job list is based on 
. ^. / \ 

the questionable assuiqDtion that the acquisition of knowledge of occupa- 

tional information can be inferred from'a list of occupational titles • 

In referring to the ufe of ess£^y' questio'ns^ on career planning, 'Gilrain did 

not recommend .its use for .the reasor^ that such compositions are. expensive 

'i « • 

and difficult to grade, (5Gilrain,f i973;>^ pp,' 12 'and' 13) 

J? ' ^ 

For additional^^mformation concerning measuring devices and criterion 

measures J refer- to Appendix J. This Appendij^ consists of a detailed list 

^ ^ ¥ ♦ 

of^ '"Multiple Criterion' Measures, fo^ ^Valuation of School Progr&ms," as 
developed by Metfessel and Michael' (19.67,' pp, 27, ^31-943), ' These. multiple 
ci'iterion jneasures are grouped according to the following five categories* 
(1) indicators of status or change in cognitive and affective behaviors of 
students in terms of standardized measures and scales, (2) indicators of 
status or change in cognitive and affective behavioics of students by in- 
formal or semi-formal teachqr-made instruments or devices, (3) indicators 
of status or change in student, behavior other than those measured by tests, 
jinventories, and observation scales in relation to the task of , evalua^ting 
objectives of school programs, (4) indicators qf status or change in cog- 
nitive and affective behaViors of teachers and other school personnel in 
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reljition to the evaluation of school programs, and (5) indicatofXof com- 
munity behaviors in relation .to the evaluation .of school programs. 

Suggested Data Collectidn Methods and Techniques . This* sub-division , 
consists of data collection methods and techniques which may, be used spe- 
cif ically for each of tLe four stages of evaluation earlier pre$entedj 
i*e*, the methods and techniques will be presented alid Separ^ited according 
to each of the four stages of evaluation for which they ^re appropriate. 

Context Evaluation ^ . Stufflebeam (et al, 1971, p. 221) suggests the 

following systems analysis techniques for use in context evaluation in 

\ 

describing the nature of the educational domain to be served: ^ 



Prograjn Evaluation and Review Technique. 

Action Systems Model (devised by Parse^l, 1966y. 

5ystem of Instructional Variables (developed by Haranon, 1967 
and 1968); ■ " -^ / ' 

^ 4 



\ 



He also recommended the following general types of sources to help 
determine major unmet needs and unused opportunities: 

1. Sample survey and opinionnaire techniques.- 

ft 

2. Experts 'and actors* conferences (actors' conferences refer 

to conferences of persons who are representative of those who 
operatje within a defined context, while experts' conferences 
refer to conferences of persons who have specialized -knowledge 
^ of a defined context)* - ' 

3. Standardized tests. 

4. Attitude scales. 

5. Diagnostic, surveys. 

6. School profiles. * - ^ , 
*7. Study visits to other systems. 

' 8. Surveys of research literature. 
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9^- .Visitatipn by teams of experts. • . ^ 

10,, Continuing study of funding opportunities. 

Inpii^t Evaluation . Even though "techniques for input 'evaluation are 
lackin'g," Stufflebeam (et al, 1971.,, pp. 228 and 229) suggests the follow- 
ing methods 'and sources of -'collecting infoniia'tion: 

1. Committee deliberations. 

2. Appeals. to the professional literature and employment of con-^ 
sultantl. \ 

3. Gage's (1963) Handboojc of Research on Teaching (ref^r to the 
Cajnplj)ell-Stanley chapter) for selecting the experimental design. 

4. » The Frograip Planning and Budgeting System. 

5. The tielphi Technique. 

6. The Convergence Technique. 

i , * ^ 

'7. Cost/;effectiyeness analysis. . , • , 

* 1 

8^ The PERT Technique (as validated for use in education by Cook, 
1966^1 • • . ^ 

I ' ' ' 

^ * Process Evaluation . The, techniques for collecting the needed infonna 

' i ' " \ ^ / - 

tion £or this, stage of program evaluation as suggested by Stufflebeam (et 

al, 1971, p. 231) are: 

I . ^ 

1. * Interaction" analysis.' 

2. Open-end end-of-the-day reaction sheets. 

3. "Interviews. • < ► 

4. Rating scales. 

Su 'Diaries kept by project, personnel. 

6. Semantic differential instruments. 

7. Record^ o^ staff meeting^. ^ . ^ " 

8. Up-dated PERT netwoiks. . 

"9. Suggestion boxes. . ' 
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, Prod uct Evaluation ^ According to Stufflebqam <1971, pp, 234 and 235), 
* 'Traditionally, experimental design has been the recommended strategy 
for conducting product types of evaluation." This method, whyL^h utilizes 
pre- and post-te^t measuring devices.,* is, valid, however, only i\^hen ample 
care is exercised in assigning experimental and control groups randomly. 
Descriptions cf tiie experimental l^d other reseai:ch designs, including 
the precautions that are considered necessary for their validity, were 
.earlier presented' on pages III-34 to) III-37 of this chapter. 

* m 

Instruments for Collecting Information ^ Most of the content for this sec- 

tion idll be presented in-^Appeijdices C, D, H-and I; 

*^ 

Suggested Commei^ial Tests^Available from W^or Test Publishers . Ap- 
pendix C conta'ins a descriptive list of suggested evaluation instruments 
(complete with source information). wHich may be useful in collecting in- 
formation for school-based career educatiori program^. This list was devel- 
oped by coijipositing commercial tests available from over n^ine. major test 
publishers. Appendisc C inclujie^ only 'student instruments which are ar- 
ranged in chronological order, according to various grade levels. Each ^ 
o£ the tests Have also been classified according to the type of variable 
measured. The format for tte 'Appendix provides columns in^ which each in- 
strument can be further classified, .by the individual decision -maker, in 

* M I " 

terms of the stage of evaluation for which they may be appropriate. The 

" i ' ' ' * » ^ ■ 

Stage for which^each instrument is appropriate is variably dependent upon 
the specific nature, needs and limitations of the-prpgram in which they are 
to be used, e.g., for sot^ programs, many of the tests will be utilized as 
pre- and post-test , instruments during context .and product evaluations; for 
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oth^ programs, many o£ the instruments \dir only be ust^d as 'post- test 
measures > due .to such limitations as shortage. o£ funds and inappropriate- 
ness, in terms p£ the stage tliat the progijam was in before evaluation began 
(some, programs do not begin evaliiation activities until they have advanced 

N ' I , ' , 

to the product stage of evaluation KhiCJi, consequently, prevents any poss- 



ibilities for pre7testing) / 



Suggested Tests Extracted from a Variety of Career Education Hand\)ooks , 
Guides, and Project Reports ^ j^pendix D contains a list of instruments that 

were 'extracted, frofn 'a variety of ^career education hajidbo'oks, guided, and. 

^ r \ ' - ^ ' ' * *^ ' . 

project reports. It is similar to Appendfx C,^ except that it also inc;iudes 

'/instruments Vhich may be used to, ^jieasure variables periaining to teachers^ 

counselors, school psychologists, local vocational education coordinators 

. (LVEC), administrators, and pafbnts and community* members . Information per- 
* • ^ ' * 

taining to such educational facilitators as these becomes especially rele- 
vant c^Uring the input and process^ stages eyaluatiop. However, due to 
thfe interdep^ndency of the four stages of evaluation, such jLnformati<5n is 

not exclusively important to only the input and process sta'jges. A? ws . 
• • • 

earlier ipentioned, the stage for which each type of instrument is appro- 
priate is variably dependent upon the specific nature, needs, and liinita- ^ 
tions of the program in which they are to be used. 

Appendices H and I, respectively, cpn]:ain: (1) a '*List of Test Pub- 
lishers and Distributors'' and (2) a list of Sources containing information . 

on tests, measurement and data analysis. ^ ' • * . ' 

' • • * ' *■ " ' 

^ For more information concerning specific instruments, that are most 
commonly used in 'measuring student ^o\rfh as it is affected by caVeer . . 
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education.^ refer tp the preceding section o£ this ch^ter, titled '*Sug- 
ges-ted IVpes 6f Measurement Devices and Criterion Measures'' (pp. III-37 to 

Suggested Methods l^ich May be. Useful in Analyzing ancj^ Treeing Information 

In regard to statistical analysis; Gilrain (197^, pp. 26 and 27) sug-^ 
gests that th^ following statistical techniques of vocational lhaturity ^ 
measurement be considered: ^ .means, standard deviations, trtest for in- 
dependent groups, t-te^t for correlated groups, analysis of vari^ce, and 



analysis of co-variance using pre-test as the co-variant: Gilrain further 
. elaborated by indicating, that , pre -treatment differences between the experi- 
mental and ^control groups may be described by the methqds of means, stan- 
dard deviations, and t-test for independent groups and overall F. The sta- 
tistics of these methods will also describe differences between grade level, 

' 3 , ' . *' . 

» . ' * " . / 

track, curriculum or school. In addition, he also sug'gested that the use 
of "the t-tes.t for correlated groups and the analysis of do -variance will 
determine the significance of gain resulting from the treatment" (Gilrain, . 
1973, p. ^27). ^ ' *\ " ' ^ . . f ' 

Ttiroughout t the professional literature', there exists numerous sources 
that are more than c[ualified to provide suggestions for the treatment and 
analysis of collected information, e.g., fumey and Robb (1971, pp. 71-104) 
dd^'an^ excellent job of presenting,, in a rather concise and understandal^fe 

form, methods for analysis and treatment of data. For additional 'informa- 

1, -k 

tion cbnceming methqds which may be used in the treatmeift. and analysis of 
data,^ refer to Appendix- I, "Sources of , Information *on Tests, Measurement., 
and Data Analysis." , ' ' " '■ . 
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• Suggested Procedures and Formats Which May be Used to Functionally Feed - 
l)acK the Results of Evaluation ^ ^ " \ '< 

The process of providing feedback is concerned with taking the results 
of evaluation and placing, them inr a functional form that will maximize the 
efforts of ,decisj.on -makers, evaluators,'and program implementors in their 
attempts at prpviding program iinprovement . 

Not only does every phase of a program have to be evaluated, but a ' 

feedback system also has to be developed for reporting the .variables that 

^are relevant to the progress and effectiveness of each phase, i.e.,, I)ecause 

* ■ 

evaluation is a continuqus ptocess, so is ^the process cf providing -feedback 
information."; \ . ' . 

The nature of the target audience which is to receive feedback reports 
iftust^be takea info close consideration when designing feedback proi^^res 
and formats. Selection of such feedback factors *as timing, frequency, Ian- 
. guage (jargon), content, media, and format should be dictated by the partic- . 

• ^ ular needs and pediliarities of the audience to be r.eached and informe^. 
Therefore, in order to comifiunicate information effectively, feedback in- 
formation must be reported on a level cf sophistication that is commensurate 
\\dth the audience's level of understanding. . *' ' : 

There are available, many different styles for reporting infomation. . 

i Figure XVI Cp.*III-4"6) developed by Stufflebeam, 1971, p. 206) illustrates 
this* hyi offering suggestions for usinjg media in reporting information. Fig- 
ure^Xn specifically tak^s into accQunt the relationship between such factors 
as: (1) audience,. (2) organization, and (3) the meaAs of communication used - 
oral and witten., As Stufflebeam (1971, p. 206) suggests,^ '*The diverse needs 
of the audiences and the complexities of the information to ]?e reported will 
require the use. of all potentially helpful methods of communication." In . 
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Figure XVI . , . 

Media Suggest jLcais for Use in Reporting -by Relationship of- the 
AudieKce to the Organization and Me^s of Communication used 



Internal 



External 



Oral 



, Written 



Reports tb committees 

One-to-one feedBack 
Tace-to-face 
'Telephone 

Professional staff meetings 
'Television 
•Radio ' 



Requested evaluation reports 

Ad hoc project evaluation 're- 
ports: Interijn and final 

Office bulletins and memoranda 

Position papers 

-Required system reports 



* 

Television reports* 
Radio rep(>^:ts 

* 

Newspaper interviews 

Spe^eches 

Civic groups 
•State Department 
•Professional meetings 
•P^T.Ai 

\ • 

Reports to the public* via 
Board -of Education meetings 



Mandated Federal -State reports 

Public reports 
» 

Pupil profile^ ^ ' 
School profile 

School newsletters, bulletins 
Press releases 
Occasional papers 

Professional publications 

Reports for demonstration to . 
other local -education agencies 
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addition, Stufflebeam (1971,. pp. 205 and 206) further suggests that re- 
gardless of who the audience is, ''Graphic presentations should be included 
to facilitate, understanding of data and informtion,. especially in public 
documents." * - - 

For suggestions pertaining to alternative types of formats that may, 

' ^ . •/ ' 

be used to organize and presen.t the results of evaluation, refer to Ap- 

• pendices F and'k'. Together, these appendices illustrate ten different . 
tables of content, most of iv'hich were talcen £rom actual third party eval- 
uations. The table of contents -illustrated'^in Appendix F, developed ^y' 
Stufflebeam (1971, pp. 200-205) was based on a work breakdo™ of evalua- 
tion tasks and, therefore, does not allow for a section providing the Je- 
suits of the evalulation. Consequently, Stufflebeam (1971, p. 205) has' 
suggested that the foilo^dng type oi sections be added to the outline: 

4, * Evaluation Results (concerning decisions to be made) 

• ^ - •} 

4.1 ^Questi^ to be answered 

4.11 Evidence 

4.12 Interpretation of evidence by decision rules 

4.2 Questioh to be ans^vered 
f 4.21 Evidence 

» 

4.22 -Interpretation of evidence by decision rules 

4.3 . /• / 



I 



! 

5. Suiiffi^ry of Evaluation (based on one or more of the followitig 
questions): 

5.1 ,^Vhat are the needs and priorities*' to be met? 

' 111-47 o 
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* 5.2 IVhat are th6 alternatives for meeting these needs and 
priorities and their relative potential for reducing 
/ needs?- 

5.3 Was the program /ivhich Kas iinplemented congruent to the 
planned program? 

' ■ ' 5.31 In what "ways did it differ? 

5.32 ^'hy.?. ^ . '. , 

5. S3 facilitants and restraints were encountered? 

5.4 How well did the progr^ meet the needs for which it was 
identified? ^ 

Suggested Criteria TOich May be Useful in 'Assessing the Adequacy of . ^ 
Evaluatipn . - 

TMs section is devoted to t\co questions: -(1) how does one evaluate 
an evaluation and (2) to what criteria must an evaluation conform, in order 
to be functionally effective? One way of evaluating an evaluation is to 
assess it in terms of the infoimatioh it provides. This method is based 
on the assumption that an.* evaluation which provides effective information 
is an effective evaluation. 

Using the effectiveness of information as a basis or significant 
.^factor for assessing .evaluation, Stufflebeam, (1971, pp. 27-32) has pro- 
posed a viable ans^^?er to question number two just listed. The criteria 
that he feels. are appropriate for evaluating the evaluation are as follows. 

i 

Scientific Criteria 

Internal Validity ^ 

^Internal validity refers to how close the relationship is between 
the collected information and the actual^ phenomena it is supposed to rep- 
resent. (Stufflebeam,, et al, 1971, p. 27) ^ 



/ 
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External Validity . * 

External validity refers to how representative -the information is. 
It pertains to whether or not the collected information , can be applied 
or generalized to larger groups or populations. (Stuff lebeam, et al, 1971, 

Reliability 

Reliabilitv refeis to how consistent the information would be if it 
were collected repeatedly, using the same methods and instruments. (Stuff le- 
beam, et al, 1971, p, 28) 

Obj activity * 

Does the evaluation system allow for the handling and treatment of 
all variables in a manner vMch is independent of personal reflections or 
feelings? (Stuff lebeam, et ai,, 197l\ p. 28) - 



Practical, Criteria 



' Relevance » ' _ ' . • 

This criterion concerns itself with 'H^rhether or not the purposes are 
rin fact, served." (Stufflebeam, al, 197,1, p,, 28) ^ 

Iinportance * - ^ - - 

This criterion is somewhat' self-explanatory. Because evaluation is 
concerned with collecting information, there must be. criterion decide, 
for the sake of brevity and conciseness ,' what is to be rjeported. Importance 
is theiTused as a ^criterion to Separate* significant and' relevant infQma- 
tion from that^yhich is insignificant and irrelevant. (Stufflebeam, et al, 
1971, p. 29) 

111 ^ 
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Scope • ^ 

This pertains to the judgmental criterion of deciding the "range of 
information'' which is to be Supplied, e^*, the extent in which the var- 
'.itius types of information are needed. (Stufflebeam, et al, 1971, p. 29) 

» '* • 

Credibility , 

This^ "relates to the quality of trust or belief" in the evaluation 
design, the evaluation information and the evaluator. (Stufflebeam, et al 
1971, p. 29) 

Timelines^ 

This refers to whether or not the information obtained from the eval 
uation is reported when it is needed and useful . . . on time. (Stuffle- 
Ibeam, et ai, 1971, 'p. 29) ' ' ' * ^ 

Pervasiveness 

This criterion refers to whether or not the findings o:^ the evalua- 
tion were reported to and .used by all persons who need to use and Icnow 
about ^ them- (Stuff lebeam, et al, 1971, p. 30) " ^ 

Prudential Criterion , 

Efficiency 

If 

Efficiency refers to the extent to which all evaluative operations 
were carried out in texms of appropriate and' thrifty use of time, cost 
and personn^. (Stufflebeam, et al, 1971, p. 30) 

i 
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